🔍 Read the full analysis: Astra’s AI Repair: Editing Model Weights Without An IQ Leap on ThorstenMeyerAI.com
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A November 2 blog post by four researchers affiliated with MIT CSAIL reports that GPT-6 Astra edited two layers of the open-weight Qwen3.5-0.8B model, raising its MATH-500 score from 7.9% to 48.1%. The result came from an iterative search using 50 model forward passes, and 33.1% of the edited model’s answers still hit the benchmark’s token limit. The work is an early, non-peer-reviewed report, not evidence that the model gained a general mathematical ability.
Four researchers affiliated with MIT CSAIL report that OpenAI’s GPT-6 Astra edited the weights of the open model Qwen3.5-0.8B, raising its score on the MATH-500 benchmark from 7.9% to 48.1% without using gradient descent. The November 2 blog post says the edit targeted a repetitive reasoning behavior, but the result remains an early, non-peer-reviewed experiment—and 33.1% of answers still reached the token limit.
The post, titled “Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights,” was written by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. Its authors say Astra identified a direction associated with looping in two layers of Qwen3.5-0.8B and subtracted it from selected weight columns. The change affected 131,072 weights, or 0.016% of the model’s 0.8 billion weights, according to the researchers.
The reported MATH-500 result came after 50 forward passes and an iterative process that took 20.3 hours. The logs record 158 turns and 80 Python runs. The researchers prohibited gradient descent, backward passes and fitting weights to a loss function; Astra instead used test feedback and a Python environment to inspect and modify the weights. The benchmark questions used for the final scores were not shown to Astra, according to the post.
The improvement was not limited to MATH-500 in the reported tests. On GSM8K, the score rose from 13.8% to 67.1% after 50 passes; on IFEval, it increased from 21.0% to 62.3%. The 10- and 25-pass runs were much less effective on MATH-500, scoring 12.3% and 14.1%. These are results reported by the authors under their test setup, not independent evaluations.
A Repair, Not a New Math Ability
The experiment matters because it suggests a model with access to open weights may be able to help diagnose and alter a specific failure mode in another model. That could give researchers and operators another way to modify open models, where the underlying parameters are available, without running a conventional training process. The reported edit was small in proportion to the full model, but required extensive testing and feedback.
The distinction between repairing a weakness and adding a new capability is central to interpreting the score. The source article notes that Qwen’s model card already warns of a tendency to enter thinking loops. The researchers themselves caution: “Fixing loops, however, is not the same as making a model better at math.” The edited model may have scored better because it spent less of its response repeating itself, rather than because the edit created stronger mathematical reasoning.
There is also a trade-off to track: the loop rate fell substantially, but did not disappear. The share of MATH-500 answers reaching the 32,768-token limit dropped from 93.0% to 33.1%. For anyone considering similar model edits, the result points to a possible targeted intervention—not a general substitute for training, evaluation or safeguards.
Popular right now
Ninja Luxe Café Mini Plus Espresso and Drip Coffee Machine with AutoFroth, Stainless Steel (ES351)
Breville Barista Express Espresso Machine, Black Sesame
LG 85-Inch QNED73B Mini LED 4K Smart TV (2026)
As an affiliate, we earn on qualifying purchases.
How the Weight Edit Was Tested
The viral framing described the result as AI improving AI “without training” and suggested it took only 50 test runs. The published logs give a more specific account. “Without training” means no gradient descent in this experiment; Astra still carried out an iterative search, used practice problems and received feedback from model runs. Its budget included 50 forward passes, each able to evaluate up to 512 problems or read up to 1,024 texts, plus a limit of 300 code runs.
The researchers tested three budgets—10, 25 and 50 forward passes. For MATH-500, those runs scored 12.3%, 14.1% and 48.1%, respectively, against 7.9% for the unedited model. The authors also report that a model without the looping problem did not improve in the same way, and that edits to Gemma 3 270M changed scores by no more than about a point. Those findings suggest the method’s success may depend on the particular failure being targeted.
The blog post describes eight experiments, not all of them successful. In one, Astra could not determine from weights alone which of two fine-tuned Gemma models had been trained to flatter users. The authors call the results “very early” and say the capabilities remain limited. The post is not on arXiv and has not been peer-reviewed, according to the source material.
““Fixing loops, however, is not the same as making a model better at math.””
— The researchers, in “Sparks of Weight-Space Programming”
Limits of the Reported Results
The findings come from a single blog post and have not been peer-reviewed, according to the source material. The published logs and edited model on Hugging Face provide details that can be inspected, but they do not establish that the result will reproduce across other models, tasks or evaluation settings. The article also reports no statement from OpenAI about the experiment; OpenAI is not listed as a co-author.
It remains unclear how much of the score increase reflects better mathematical reasoning versus fewer wasted tokens, and whether the edit affects other behaviors or performance on tasks not reported. The post does not establish that similar edits can reliably produce gains with smaller budgets: the 10- and 25-pass MATH-500 results were modest, while the 50-pass run required 20.3 hours. Independent replication and broader testing would be needed to assess the method’s reliability.
Replication and Broader Model Tests
The immediate next step is independent replication of the Qwen3.5-0.8B edit, including checks of the reported benchmark scores, token-limit rates and performance on tasks beyond those described. The researchers’ post and published logs offer a basis for inspection, but the source material does not announce a peer-reviewed paper or a scheduled follow-up evaluation.
Further tests could clarify whether the technique works on other models with similar looping problems, whether it can be carried out with fewer forward passes, and whether removing a behavior causes unwanted side effects. Until such results are available, the reported gains should be treated as specific to this experiment, rather than evidence that AI systems can routinely improve other models’ abilities by editing their weights.
Key Questions
What did GPT-6 Astra change?
According to the researchers, Astra identified a direction linked to repetitive outputs and subtracted it from selected weight columns in layers 15 and 19 of Qwen3.5-0.8B. The edit changed 131,072 weights, or 0.016% of the model’s total weights.
Did the edited model score 48.1% on every math test?
No. The 48.1% score was on MATH-500 after the 50-pass run. The researchers separately report 67.1% on GSM8K and 62.3% on IFEval after that run; these results come from the post’s stated setup.
What does “without training” mean here?
It means the researchers prohibited gradient descent, backward passes and fitting the weights to a loss. Astra still used repeated model evaluations, practice questions, Python runs and feedback to search for an edit.
Did the edit eliminate the model’s looping?
No. On MATH-500, the share of answers hitting the 32,768-token limit fell from 93.0% to 33.1%, according to the reported results. A substantial share still reached the limit.
Has the result been independently verified?
The source material describes the work as a non-peer-reviewed blog post and says it is not on arXiv. The logs and edited model are available for inspection, but independent replication is not reported.
Source: ThorstenMeyerAI.com
