AIM reaches strong Flash Attention solutions earlier
On Flash Attention, AIM reaches 90.5% at 3.3 hours; ScientistOne reaches 86.2% at 3.5 hours. The authors report up to 3.1× faster time to the strongest baseline’s score.
Left: mean best-so-far AutoLab score (%) versus elapsed wall-clock hours; right: labeled endpoint scores/times and AIM times reaching baseline levels. All methods use Gemini-3.1-Pro-Preview; up to 300 executions and six hours for this task. Three independent runs; Table 1 reports mean and standard error.
View original chart and methods ↗
AIM: Agentic Idea Management for Automated Research · Figure 4 (HTML container S5.F5); Section 5.1; Table 1
How to interpret this chart
- This figure concerns Flash Attention only; the 3.1× claim is not a suite-wide research speedup.
- Normalized AutoLab score is not percentage runtime reduction. No uncertainty bands are shown in this time plot.
- Equal nominal execution/time caps do not imply equal tokens or monetary costs.
- The authors designed the management scaffold; this does not demonstrate autonomous improvement of the optimizer itself.
Published values from the chart
Exact labels in Figure 4. Endpoints differ in quality; do not divide endpoint times to reproduce the matched-score 3.1× claim.
| Method | Printed endpoint score | Printed time |
|---|---|---|
| AIM | 90.5% | 3.3 h |
| ScientistOne | 86.2% | 3.5 h |
| AdaEvolve | 85.3% | 5.5 h |
| AIRA (MCTS) | 77.9% | 3.7 h |
| Arbor (depth=3) | 77.2% | 2.2 h |
| DeepScientist | 72.3% | 5.0 h |