Decision. Retain the current prediction model. A compact neural reranker can run quickly in native R, but this controlled trial does not improve accuracy. The unchanged model reaches 164/600 top-three matches (27.3%). The two new training methods average 162/600 (27.0%) and 161.3/600 (26.9%) at their selected epochs across three seeds. The product preview adds a clearer identity, an optional list of further suggestions and an illustrated guide. These interface improvements are distinct from model accuracy.

In plain language, top three means the recorded next word appears among the first three suggestions. A reranker changes their order. The teacher is a larger neural model used during research. A seed changes the random starting point of a training run; repeating the run checks whether the result is stable.

Publication context. This report preserves the English-model experiment of 26 September 2026. The later PhraseFlow 1.4 app adds languages and interface features; it does not deploy the unsuccessful reranker. The current user guide documents that release. The experimental results below are unchanged.

Question and design

Can a small model use more of the preceding sentence to improve the order of existing suggestions while retaining fast R/CPU inference? The previous diagnosis found that a frozen neural teacher reached 204/600 top-three matches (34.0%) when reordering the exact same 128 candidates, versus 164/600 for the CPU model. This provides a measured opportunity to investigate, not a guarantee that a smaller model can reproduce the teacher.

The registered trial fixes the CPU model, 50,000-word vocabulary, normalization, 128 prefix-generated candidates and 600 reused development cases, equally split among blogs, news and Twitter. It uses 3,000 existing training prefixes with no overlap with those development cases. The reserved 900 examples were not opened in this study. Quiz questions never enter training or model selection.

A frozen, previously trained GRU encodes up to 48 words. A 12,801-parameter scoring head combines this context, the candidate word representation, their interactions and six simple features. Its final layer starts at zero so that initial predictions match the CPU model exactly. Scaling is fitted using training candidates only. The teacher remains the existing Qwen3-1.7B-Base with its fixed local adapter and a 128-model-token limit.

Two equal-budget objectives are compared with three fixed seeds and 12 epochs:

  • Target only: learn from the recorded next word when it is already in the list.
  • Conditional teacher: combine that objective with 25% teacher guidance, normalized within the same candidate list at temperature 2. The conventional temperature-squared factor is applied.

No correct word is inserted into a list. The recorded target is present in 1,710/3,000 training lists and 332/600 development lists. Missing training targets receive no fabricated label. The teacher-guided objective can still learn from their candidate scores.

Results

The epoch for each method is selected by mean development top-three accuracy across seeds, with predeclared tie rules. All seeds are retained. These are exploratory development comparisons, not independent test results.

Table 1. Matched development results; means across seeds can be fractional.
Method Epoch First correct (mean) Within three (mean) Top-three counts by seed
Unchanged CPU NA 106/600 (17.7%) 164/600 (27.3%) Deterministic baseline
TargetOnly 1 98.3/600 (16.4%) 162.0/600 (27.0%) 164, 158, 164
ConditionalTeacher25 8 100.7/600 (16.8%) 161.3/600 (26.9%) 161, 161, 162

Neither method meets the accuracy requirements of at least 170 top-three matches on average, at least 106 first-word matches on average, two seeds improving top three and no seed losing first-word accuracy. The best method by the registered selection rule is target-only at epoch 1; its fixed representative seed has 99 first-word matches and 164 top-three matches. It gains 11 top-three successes and loses 11. Its paired difference is 0.0 percentage points, with a descriptive 95% interval of -1.5 to +1.5 points.

All seeds and paired uncertainty
Method Epoch Seed First correct Within three Gained Lost Difference (percentage points)
TargetOnly 1 20260926 99 164 11 11 0.0 [-1.5, 1.5]
TargetOnly 1 20260927 98 158 10 16 -1.0 [-2.7, 0.7]
TargetOnly 1 20260928 98 164 11 11 0.0 [-1.5, 1.5]
ConditionalTeacher25 8 20260926 103 161 14 17 -0.5 [-2.3, 1.3]
ConditionalTeacher25 8 20260927 99 161 15 18 -0.5 [-2.3, 1.5]
ConditionalTeacher25 8 20260928 100 162 14 16 -0.3 [-2.2, 1.5]
Intervals use 10,000 paired bootstrap resamples stratified by source. They compare the same cases. They do not correct for epoch selection or repeated use of development data and therefore cannot establish a confirmatory improvement.
Results by text source and context length
Grouping Group Examples Target in list Original top three Selected reranker top three
source blogs 200 57.5% 26.0% 26.5%
context_length 1-4 words 188 54.8% 29.8% 31.4%
context_length 5-12 words 171 58.5% 26.9% 26.3%
context_length 13+ words 241 53.5% 25.7% 24.9%
source news 200 51.5% 27.5% 28.5%
source twitter 200 57.0% 28.5% 27.0%
These describe the fixed representative run. Small subgroup differences on reused development examples do not establish a reliable improvement. Counts for every seed are retained in subgroup_results.csv.

What the learning curves show

Figure 1. Training success rises while development success does not. Each line is one fixed seed; epoch zero reproduces the CPU baseline.

Figure 1. Training success rises while development success does not. Each line is one fixed seed; epoch zero reproduces the CPU baseline.

The extra training increasingly fits the training examples without improving the development cases. This is evidence of an overfitting pattern within this trial. It does not establish that all longer-context models are ineffective. The small continuation sample and frozen, modest context encoder remain plausible limitations; this experiment does not isolate their individual contributions.

Unlike the earlier full-vocabulary distillation objective, the new teacher loss explicitly compares relative ordering within the same list. Even at its best mean development epoch, that change does not transfer the teacher’s advantage. Better loss alignment is therefore insufficient on its own in this implementation.

Figure 2. Conditional teacher divergence on development data. A lower value means closer agreement with the teacher within the candidate list; it is not accuracy or calibrated confidence.

Figure 2. Conditional teacher divergence on development data. A lower value means closer agreement with the teacher within the candidate list; it is not accuracy or calibrated confidence.

Speed, memory and reproducibility

The selected representative model produces the same top-three words in Python and native R on all 600 development cases. The maximum score difference is 0.000220; rankings agree despite floating-point differences. An R feature expression was corrected during export verification before the final successful check; the Python training results were unaffected.

Native R complete inference, including CPU candidate generation, context encoding and reranking, takes 2.66 ms median and 4.51 ms at the 95th percentile, across 1,800 warm calls. Combined R objects occupy 240.8 MiB. These meet the declared CPU/memory limits, but do not compensate for the failed accuracy requirement. The separately recorded 0.58 ms production benchmark uses a different timing run and a simpler three-word lookup path; this is not a controlled speed-ratio experiment.

Teacher scoring of the 3,000 training prefixes took 391.1 seconds on the GPU. The six small-head runs, including evaluation, took 9.1 seconds on the GPU, after feature preparation. These figures exclude process startup and are not app response times.

The study preserves its protocol, input/checkpoint fingerprints, all epoch checkpoints, component losses, case-level results and native R export. Raw text, quiz details and model weights remain in private project data folders. The production model checksum remains unchanged.

Reproduce the study in a fresh project copy

The scripts intentionally refuse to overwrite completed evidence. Retain the input models and private training/development files. Run from the corpus project root in a fresh study copy, then knit this R Markdown report. No step requires opening the reserved test examples.

source('final_project/research/phraseflow-20260926/prepare_training.R')
# Python, using the saved project environment:
# python score_training_teacher.py
# python train_reranker.py
source('final_project/research/phraseflow-20260926/evaluate_r.R')
# Optional quiz checks were run privately after freezing the decision.
# Only their aggregate results are included in the public package.
Exact paths, environment and output definitions are in the accompanying README and R-session-info.txt. This HTML report is knitted from R Markdown using saved measurements, not fabricated figure values.

Quiz demonstrations and natural wording

The two previously supplied ten-question quizzes were checked after the model decision was frozen. The table compares the current CPU app with the saved reviewed/submitted choices. The archived neural option rankings come from earlier case studies, not a new teacher run. The already exposed items are not an unbiased measure of general language prediction.

Table 2. Agreement counts on ten previously exposed items per quiz. Question and answer text remains private.
Quiz Items CPU first word CPU within three CPU supplied options Archived neural options
2 10 4 5 6 9
3 10 1 2 4 8

The recorded course grades of 100% describe the submitted, reviewed quiz responses. They do not mean that the currently deployed CPU model independently predicts every answer. Free prediction and ranking four supplied options are different tasks.

Naturalness also differs from exact matching: two plausible continuations may express different intentions. A future semantic evaluation should use unseen prefixes, independently assessed acceptable continuations and blinded ratings. It should report exact-match accuracy alongside appropriateness, rather than relabeling plausible mistakes as correct after inspecting outputs.

Current literature and design choices

The literature check was performed on 26 September 2026 using primary publications. Recent methods are considered for their relevance to this task and hosting budget, not simply their publication year.

Source Status and relevant contribution Implication here
Qwen Team, 2025 [1] Organization-authored technical report for pretrained language models The existing frozen Qwen3 teacher supplies a consistent comparison. Its benchmark scores do not transfer to this app.
Li et al., ACL 2025 [2] Peer-reviewed study of training-sample influence in next-word prediction Supports examining representation and generalization, rather than optimizing only familiar examples. This study does not replicate their method.
Wei et al., July 2026 [3] Preprint on separately scaling pretrained memory, including billion-parameter memory modules A relevant longer-term research direction, but its resource needs and different benchmarks do not establish suitability for this R/CPU deployment. Not implemented here.
Hugging Face, SmolLM3, 2025 [4] Organization release of a 3-billion-parameter language model A possible future teacher comparison, not a lightweight drop-in replacement under the present native R memory limit. Not downloaded or evaluated.

Discussion and conclusion

The product now has a clearer user guide, explicit document metadata and an optional longer suggestion list. Tests confirm that these changes preserve the original first three predictions on all 600 development cases. They improve usability and documentation, not measured predictive accuracy.

The modeling conclusion is equally specific: retain the current CPU predictor. Correcting the distillation objective and adding context-by-candidate interactions did not yield a robust accuracy gain. The measured candidate recall of 55.3% is the maximum exact-match success achievable by a reranker restricted to these lists on these 600 cases; the teacher’s 34.0% top-three score is a measured result, not a theoretical ceiling. Perfect prediction is not supported.

The next justified modeling experiment is to improve the context representation while keeping the corrected candidate-conditional loss and CPU shortlist fixed. Compare a lightly unfrozen encoder with this frozen control using a larger, source-balanced training sample and a preregistered learning curve. That would test representation and training-scale limitations explicitly. Any candidate must pass CPU limits and a new independent evaluation before deployment. Confidence calibration follows model selection on separate data. This follow-up has not been run.

Final-project requirements

The course submission requires the Shiny app URL and a presentation of at most five RStudio Presenter slides on RPubs. The portfolio and this research report are supporting publications, not additional items for peer grading. PhraseFlow 1.4 is now published with 5, 10 or 20 word choices while retaining one prominent next word. It preserves the English word model studied here. The course submission remains pending author review. Current illustrated guide · Study source and measured results.

Requirement Evidence
Shiny app on shinyapps.io Public app is published as PhraseFlow 1.4.
Phrase input and one next-word prediction Prominent first suggestion; optional alternatives. Published app and local server tests pass.
Five English news/Twitter prefixes produce predictions Live release check on 27 September: 5/5; English model unchanged. This is availability, not correctness.
No more than five RStudio Presenter slides on RPubs Published five-slide pitch, checked without signing in. Native .Rpres source retained.
Algorithm, quantitative performance and usage Explained in pitch, Results tab, current illustrated HTML guide and this study.
Experience, originality and hiring judgment Subjective peer assessment; no grade guarantee.

References

  1. Qwen Team (2025). Qwen3 Technical Report. Technical report / preprint.
  2. Li, Y., Du, Y., Liu, Y., Feng, F., Feng, M. X., & Wu, Y. (2025). On Support Samples of Next Word Prediction. ACL, 10277-10289. doi:10.18653/v1/2025.acl-long.507.
  3. Wei, R., et al. (2026). Memory Decoder at Scale: A Pretrained, Parametric Long-Term Memory. Preprint, submitted 30 July 2026; no peer-reviewed acceptance established in this review.
  4. Hugging Face (2025). SmolLM3: smol, multilingual, long-context reasoner. Organization-authored model release.
  5. Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the Knowledge in a Neural Network. Foundational distillation method.
  6. Chen, S. F., & Goodman, J. (1996). An Empirical Study of Smoothing Techniques for Language Modeling. ACL. Statistical model foundation.