Some artificial intelligences (AIs) are capable of changing the programs and instructions they run on in order to improve results. But such edits can introduce errors, and checking unsuccessful changes consumes resources. A group of scientists from the University of Virginia, Microsoft Research and Tsinghua University has developed a forecasting program that helps AI agents foresee the consequences of proposed edits before testing the modified code. The paper was published on the arXiv preprint server.
The approach was tested on various tasks: programming, predicting the acceptance of scientific papers for publication, developing rewards for simulated robots, and evaluating solutions to complex mathematical problems. Preliminary results show that using forecasts helps agents improve more efficiently.
The method is based on the HyperAgents system, which combines two AI components. The first performs tasks, the second modifies the code and instructions used by both. Previous versions and their results are stored in an archive so they can be taken into account in subsequent changes.
The new component, an improvement dynamics model, analyzes the proposed edit, the agent being modified and past results. It predicts the outcome for each task, distinguishing incorrect answers from attempts that were not completed. This matters: an incorrect answer may require better reasoning, while an unfinished attempt may require fixing the program.
If the forecast, adjusted for the previous prediction error, shows that the result will be worse than the current measured one, the component making the changes receives this forecast and one more opportunity for revision. The final agent is tested independently of the final forecast. The edits affect its program and instructions, while the weights of the base language model remain unchanged.
In one example, an edit moved instructions outside quotation marks, which broke the program. The modifier agent noticed the error but exhausted the available tool calls before fixing it. The predictor predicted unfinished tasks, which prompted another revision. After fixing the quotation marks and checking the syntax, the updated agent completed 11 tasks.
The researchers conducted five runs with 40 improvement rounds for each experimental condition. The forecasting approach produced stronger programmer agents faster than HyperAgents. On 166 programming tasks separated from the improvement and selection process, the selected agents showed a median success rate of about 30% versus 22% for HyperAgents and 16% for the common starting agent. Success meant creating a software patch that passes the tests for a specific task.
When compared with equal budgets for language model tokens (units of text processed or generated), the forecasting approach retained its advantage. Tokens for editing, forecasting, forecast updating and task evaluation were counted, but total computational costs were not measured.
The forecasting program can also improve. Every five rounds, the system proposes updates and compares their predictions with subsequently recorded results before deciding whether to accept them. The team also checked whether the experience gained could be transferred to new tasks. A predictor trained jointly on predicting paper acceptance and developing rewards for simulated robots was applied to evaluating mathematical solutions. On 100 unseen tasks, the selected agents exactly matched expert assessments in a median of 55% of cases. Agents with a fresh predictor reached 51%, and those with HyperAgents reached 47%.
These results do not prove that AI agents can replace human reviewers, evaluators or programmers. They only suggest that forecasting helps agents find useful changes in programs, and that the experience of such predictions may be useful when improving on another task.