Reproducibility Report for ‘Base Models Know How to Reason, Thinking Models Learn When’ by Venhoff et al. (arXiv preprint: 2510.07364v3, 2025)
Introduction
The target study introduces a hybrid reasoning framework in which a base language model is selectively steered at inference time to exhibit reasoning behaviors characteristic of a “thinking model.” The core claim is that base models already contain latent reasoning mechanisms, and that post-training primarily teaches when to activate these mechanisms rather than how to perform them.
The approach operationalizes this idea using steering vectors: learned directions in the base model’s activation space that causally induce specific reasoning behaviors (e.g., arithmetic execution, backtracking, uncertainty estimation). A hybrid inference procedure then combines a thinking model and a base model, using signals from the thinking model to decide which steering vector to apply at each token while the base model generates the output.
This reproduction project focuses exclusively on the two central algorithmic components of the paper: (1) Training steering vectors for a base model using thinking-model supervision; (2) Training and evaluating the hybrid generation system that applies these vectors during inference.
All other components described in the original paper (e.g., taxonomy discovery via sparse autoencoders) are treated as fixed inputs to these two stages and are not themselves reproduced.
Justification for choice of study
My PhD research focuses on the analysis of chain-of-thought reasoning in humans and language models, with particular interest in identifying and manipulating intermediate reasoning behaviors. This study is directly aligned with those goals, as it provides a concrete, mechanistic method for isolating reasoning behaviors and causally activating them in base models without parameter updates.
Reproducing this work is especially valuable because it combines mechanistic interpretability with task-level evaluation, offering a rare opportunity to connect internal representations, controlled interventions, and downstream performance within a single framework.
Anticipated challenges
- Steering effects may be weaker or noisier at smaller model scales.
- Hybrid token-level inference is computationally expensive and sensitive to hyperparameters (e.g., steering coefficients and window sizes).
- Evaluation via numerical parsing may undercount correct answers with atypical formatting.
Links
Project repository (on Github): https://github.com/psych251/venhoff2025
Original paper (as hosted in your repo): https://arxiv.org/pdf/2510.07364
Methods
Description of the steps required to reproduce the results
This reproduction implements the same two-stage pipeline used in the original study, using one of the four base models evaluated by the authors: Qwen2.5-Math-1.5B.
Step 1: Steering vector training
Steering vectors are trained for a fixed base model (Qwen2.5-Math-1.5B) using supervision from a thinking model (DeepSeek-R1-Distill-Qwen-1.5B). Each steering vector is a learnable parameter with the same dimensionality as the base model’s hidden states at a chosen layer.
Following the original study, I train a total of 16 steering vectors: 15 category-specific steering vectors, each corresponding to a distinct reasoning behavior (e.g., arithmetic execution, backtracking, uncertainty estimation), and one general bias vector capturing global stylistic and structural properties of thinking-model outputs.
For each reasoning category, training examples consist of prefixes and target continuations derived from thinking-model reasoning traces. During optimization, the steering vector is added to the base model’s hidden states, and its parameters are updated to minimize cross-entropy loss on the thinking model’s target tokens, while keeping all base model weights frozen.
Step 2: Hybrid model training and evaluation
The hybrid model combines the base and thinking models during inference. At each token position, the thinking model provides a signal indicating which reasoning behavior should be activated. The corresponding category-specific steering vector (together with the bias vector) is applied to the base model’s activations.
Multiple candidate continuations are evaluated per token using a perplexity-based guardrail under the thinking model. Hybrid performance is evaluated by comparing base-only, hybrid, and thinking-model outputs on mathematical reasoning benchmarks.
Differences from original study
- Model: I use the same base/thinking model pairing (Qwen2.5-Math-1.5B with DeepSeek-R1-Distill-Qwen-1.5B) and the same benchmark target (GSM8K) as in the original study, so this part is matched completely.
- Compute environment: Experiments are run on NVIDIA A40 GPUs rather than the authors’ hardware. This may affect runtime and the practical breadth of sweeps (e.g., fewer coefficient/window settings explored), but should not change the core algorithmic procedure.
- Implementation details / defaults: Where the paper leaves minor implementation choices implicit (e.g., smoothing/plotting conventions, logging, or exact sweep granularity), I follow the released repo scripts and my own engineering defaults; these choices may cause small numerical differences but should not change qualitative behavior.
- Evaluation method: The original paper does not specify the exact method used to evaluate answer correctness. The released codebase supports two evaluation options: (1) LLM-as-a-judge, which compares the model’s full answer against the gold solution using a strong external model (GPT-4.1 is used by default in the repository); (2) Numerical parsing, which extracts and compares final numeric answers directly. Due to financial constraints (since LLM-based evaluation would require judging several thousand model outputs at significant cost) I use the numerical parsing evaluation method. While this approach is deterministic and cost-effective, it may differ from LLM-as-a-judge evaluation in cases where answers are correct but formatted atypically, or where reasoning is correct but the final numeric expression is ambiguous. This difference may lead to quantitative discrepancies relative to the paper’s reported accuracies, but should not affect the qualitative comparison between base, hybrid, and thinking models.
Project Progress Check 1
Measure of success
Reproduction success will be measured by replicating the end-to-end hybrid model evaluation reported for the Qwen2.5-Math-1.5B base model paired with DeepSeek-R1-Distill-Qwen-1.5B as the thinking model, on GSM8K benchmark.
The primary outcome measure is accuracy (%) of the three models:
- Base: unsteered base model accuracy
- Hybrid: token-level hybrid steering accuracy
- Thinking: thinking model accuracy
Success will be evaluated by comparing reproduced accuracies to the paper’s reported results for this model pair. For reference, the paper reports the following GSM8K accuracies (see figure from the paper below):
- Base: 83.8%
- Hybrid: 80.8% (−3.0%)
- Thinking: 80.8% (−3.0%)
- Gap recovery: 0.0%
It is important to note that this model pair corresponds to one of the four base models evaluated in the original study, and is also the smallest model configuration considered. In the paper, this setting is the only one in which the hybrid model does not outperform the base model, and the reported results show no gap recovery for this pair. Consequently, reproducing this behavior is not indicative of a failure of the method, but rather reflects a known limitation of the approach at smaller model scales.
Due to computational constraints, this Qwen2.5-Math-1.5B configuration is the only model setting that can be fully reproduced within the scope of this project. As such, this reproduction focuses on matching the qualitative behavior and relative performance relationships (base vs. hybrid vs. thinking) reported for this model, rather than demonstrating gains that only emerge at larger scales.
Pipeline progress
Steering vector training: I have successfully trained steering vectors 1–3 for the base model (Qwen2.5-Math-1.5B) using the training pipeline from the paper. Across all three vectors, the training objective decreases consistently over optimization iterations, reflecting the expected convergence behavior and indicating that the steering-vector optimization procedure is functioning correctly (see training loss curves for each of the three vectors below). This is the first major step towards obtaining all 15 steering vectors, which will then be used to develop the hybrid model, and evaluate it on the GSM8K benchmark.
Results
Data preparation
Key analysis
Side-by-side graph with original graph is ideal here