FrontierSWE v2 | RSI Progress

Why it matters here

FrontierSWE v2 evaluates long-horizon technical work across mixed engineering, scientific-computing and AI-research tasks. Its six-task harness ablation illustrates how scaffolding and stopping behavior influence measured scores. This draft retains that methodological context without treating the aggregate as an isolated AI-R&D or recursive-improvement result.

AI-R&D capabilityEvaluation integrity

What the charts show

FrontierSWE v2: harness comparison on six selected tasks

The authors report higher average rewards with Proximus across the six-task comparison. The plotted task rewards improve for Opus 5 on each selected task; GPT-5.6 Sol has mixed task-level changes, including slightly lower Astronomy and SPICE rewards.

Paired reward and mean agent runtime bars for Opus 5 with Proximus versus Claude Code, and GPT-5.6 Sol with Proximus versus Codex. Caption reports six tasks and five trials per task. This measures sensitivity to externally designed scaffolding.

View original chart and methods ↗
FrontierSWE v2 · Methodology Improvements / Proximus harness, six-task reward and mean time used figure; Limitations and Future Work

How to interpret this chart

Six selected tasks and five trials per task; this comparison is not the complete 34-task aggregate or a pure AI-R&D subset.
The maximum task budget is 20 hours, but actual time used differs substantially across harnesses; equal spent-compute, prompt, hardware and harness-version matching were not verified.
Developer-reported extraction, not independent evaluation or experimental replication. No confidence intervals are displayed for the paired task rewards.
The Proximus harness deliberately encourages longer work and exposes grading and remaining-time information; evaluation awareness limits generalization.
Scores do not establish autonomous novel research, recursive improvement or an autonomy level.

Published values from the chart

Exact printed mean reward labels, not estimates from bar lengths. Five trials per task per the figure's surrounding text. Mean runtime was inspected separately and is not a human task-duration estimate or research-speedup ratio.

FrontierSWE v2: harness comparison on six selected tasks
Task	Opus 5 + Proximus	Opus 5 + Claude Code	GPT-5.6 Sol + Proximus	GPT-5.6 Sol + Codex
Astronomy Toolkit	0.835	0.509	0.133	0.142
Remotion Video Generation	0.219	0.185	0.185	0.048
Vision-only TORCS Racing Bot	0.138	0.126	0.024	0.022
SPICE Simulator in Rust	0.758	0.593	0.431	0.434
Snooker Prediction	0.571	0.408	0.708	0.599
Music Diarization	0.538	0.526	0.449	0.397

What to keep in mind

Six selected tasks and five trials per task; this comparison is not the complete 34-task aggregate or a pure AI-R&D subset.

The maximum task budget is 20 hours, but actual time used differs substantially across harnesses; equal spent-compute, prompt, hardware and harness-version matching were not verified.

Developer-reported extraction, not independent evaluation or experimental replication. No confidence intervals are displayed for the paired task rewards.

The Proximus harness deliberately encourages longer work and exposes grading and remaining-time information; evaluation awareness limits generalization.

Scores do not establish autonomous novel research, recursive improvement or an autonomy level.

The complete benchmark, Haiku result, per-task artifacts and uncertainty methods require further extraction before numeric benchmark records are proposed.