Benchmark
Internal Research Debugging Evaluation
Benchmark by OpenAI
Astra card snapshot, September 2026
78.05%
GPT-6 Astra
frontier lab
OpenAI
Current AI self-improvement suite plus original MLE-bench and PaperBench. Graph-only scores withheld.
Published evidence
Benchmark
PaperBench
Benchmark by OpenAI
Original paper v1
2.6 ± 0.2%
o3-mini-high / BasicAgent
Benchmark
PaperBench
Benchmark by OpenAI
Original paper v1
4.1 ± 0.1%
GPT-4o / BasicAgent
Benchmark
PaperBench
Benchmark by OpenAI
Original paper v1
13.2 ± 0.3%
o1-high / BasicAgent
Benchmark
PaperBench
Benchmark by OpenAI
Original paper v1
8.5 ± 0.8%
o3-mini-high / IterativeAgent
Benchmark
PaperBench
Benchmark by OpenAI
Original paper v1
24.4 ± 0.7%
o1-high / IterativeAgent
Benchmark
PaperBench
Benchmark by OpenAI
Original paper v1
26.0 ± 0.3%
o1-high / IterativeAgent, 36 h
Benchmark
MLE-bench
Benchmark by OpenAI
Original paper v1 (2024)
16.9 ± 1.1%
o1-preview / AIDE
Benchmark
MLE-bench
Benchmark by OpenAI
Original paper v1 (2024)
8.7 ± 0.5%
GPT-4o / AIDE
Benchmark
Task-completion time horizons
Benchmark by METR
TH1 (historical estimates)
138 minutes
GPT-5 / METR horizon evaluation
Benchmark
Task-completion time horizons
Benchmark by METR
TH1.1
214 minutes
GPT-5 / METR horizon evaluation
Benchmark
Task-completion time horizons
Benchmark by METR
TH1 (historical estimates)
94 minutes
o3 / METR horizon evaluation
Benchmark
Task-completion time horizons
Benchmark by METR
TH1.1
121 minutes
o3 / METR horizon evaluation
Benchmark
AI4AI-Bench
Benchmark by AI4AI-Bench authors
Paper v1
0.191 normalized score
GPT-5.6 Sol / Codex effort aggregate
Benchmark
AI4AI-Bench
Benchmark by AI4AI-Bench authors
Paper v1
0.135 normalized score
GPT-5.6 Terra / Codex effort aggregate
Benchmark
AI4AI-Bench
Benchmark by AI4AI-Bench authors
Paper v1
0.117 normalized score
GPT-5.6 Luna / Codex effort aggregate
Benchmark
KernelGen 1P
Benchmark by OpenAI
Astra card snapshot, September 2026
66.72%
GPT-6 Astra
Benchmark
KernelGen 1P
Benchmark by OpenAI
Astra card snapshot, September 2026
38.12%
GPT-6 Sol
Benchmark
KernelGen 1P
Benchmark by OpenAI
Astra card snapshot, September 2026
61.08%
GPT-5.6 Sol
Benchmark
KernelGen 1P
Benchmark by OpenAI
Astra card snapshot, September 2026
21.83%
GPT-6 Luna
Benchmark
KernelGen 1P
Benchmark by OpenAI
Astra card snapshot, September 2026
22.4%
GPT-5.6 Luna
Benchmark
MLE-bench
Benchmark by OpenAI
Revised, Astra card snapshot
93.80%
GPT-6 Astra
Benchmark
MLE-bench
Benchmark by OpenAI
Revised, Astra card snapshot
87.18%
GPT-5.6 Sol
Benchmark
NanoGPT
Benchmark by OpenAI
Astra card snapshot, September 2026
Qualitative result
GPT-6 Astra
Benchmark
NanoGPT
Benchmark by OpenAI
Astra card snapshot, September 2026
Qualitative result
GPT-5.6 Sol
Benchmark
PostTrainBench Lite
Benchmark by OpenAI
Astra card snapshot, September 2026
Qualitative result
GPT-6 Astra
Benchmark
PostTrainBench Lite
Benchmark by OpenAI
Astra card snapshot, September 2026
Qualitative result
GPT-5.6 Sol
Benchmark
Internal Research Debugging Evaluation
Benchmark by OpenAI
Astra card snapshot, September 2026
64.20%
GPT-6 Sol
Benchmark
Internal Research Debugging Evaluation
Benchmark by OpenAI
Astra card snapshot, September 2026
68.32%
GPT-5.6 Sol
Benchmark
Internal Research Debugging Evaluation
Benchmark by OpenAI
Astra card snapshot, September 2026
46.62%
GPT-6 Luna
Benchmark
Internal Research Debugging Evaluation
Benchmark by OpenAI
Astra card snapshot, September 2026
50.8%
GPT-5.6 Luna
Benchmark
PaperBench
Benchmark by OpenAI
Original paper v1, Code-Dev variant
43.4 ± 0.8%
o1-high / IterativeAgent, PaperBench Code-Dev
Benchmark
MLE-bench
Benchmark by OpenAI
Original paper v1 (2024)
0.8 ± 0.5%
GPT-4o / MLAB
Benchmark
MLE-bench
Benchmark by OpenAI
Original paper v1 (2024)
4.4 ± 1.4%
GPT-4o / OpenHands
Benchmark
Task-completion time horizons
Benchmark by METR
TH1 (historical estimates)
173 minutes
GPT-5.1-codex-max / METR horizon evaluation
Benchmark
Task-completion time horizons
Benchmark by METR
TH1 (historical estimates)
8.5 minutes
GPT-4 1106 / METR horizon evaluation
Benchmark
Task-completion time horizons
Benchmark by METR
TH1 (historical estimates)
5.4 minutes
GPT-4 0314 / METR horizon evaluation
Benchmark
Task-completion time horizons
Benchmark by METR
TH1.1
3.6 minutes
GPT-4 1106 / METR horizon evaluation
Benchmark
Task-completion time horizons
Benchmark by METR
TH1.1
3.5 minutes
GPT-4 0314 / METR horizon evaluation
Benchmark
Task-completion time horizons
Benchmark by METR
TH1.1, GPT-5.6 Sol June 2026 assessment
around 11.3 hours hours
GPT-5.6 Sol / METR horizon evaluation
Benchmark
PaperBench
Benchmark by OpenAI
Original paper v1, three-paper human comparison subset
26.6%
o1-high / IterativeAgent, three-paper subset
Benchmark
Research outcome prediction
Benchmark by Research outcome prediction authors
1,585 later-paper idea pairs
51.4%
Unadapted GPT-4.1, Table 4 baseline
Benchmark
PostTrainBench
Benchmark by PostTrainBench team
PostTrainBench v1.1 · September 2026 results
19.00%
GPT-5.4, High reasoning / Codex CLI
Benchmark
PostTrainBench
Benchmark by PostTrainBench team
PostTrainBench v1.1 · September 2026 results
27.23%
GPT-5.5, xHigh reasoning / Codex CLI
Benchmark
PostTrainBench
Benchmark by PostTrainBench team
PostTrainBench v1.1 · September 2026 results
36.23%
GPT-5.6 Sol, Max reasoning / Codex CLI
Benchmark
PostTrainBench
Benchmark by PostTrainBench team
PostTrainBench v1.2 · September 30 release
41.88%
GPT-6 Astra / Codex CLI, Max
Benchmark
PostTrainBench
Benchmark by PostTrainBench team
PostTrainBench v1.2 · September 30 release
31.01%
GPT-5.6 Sol, Max reasoning / Codex CLI
Benchmark
PostTrainBench
Benchmark by PostTrainBench team
PostTrainBench v1.2 · September 30 release
23.87%
GPT-5.5, xHigh reasoning / Codex CLI
Benchmark
PostTrainBench
Benchmark by PostTrainBench team
PostTrainBench v1.2 · September 30 release
17.76%
GPT-5.4, High reasoning / Codex CLI
Benchmark
PostTrainBench
Benchmark by PostTrainBench team
PostTrainBench v1.1 / Google OpenCode evaluation, October 2026
44.3%
GPT-6 Astra / Google OpenCode evaluation
Benchmark
Internal Research Debugging Evaluation
Benchmark by OpenAI
GPT-6.1 Sol card snapshot, September 29, 2026
75.52%
GPT-6.1 Sol
Benchmark
KernelGen 1P
Benchmark by OpenAI
GPT-6.1 Sol card snapshot, September 29, 2026
60.44%
GPT-6.1 Sol
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
1.60× ×
GPT-6 Astra (max) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
1.13× ×
GPT-6 Astra (max) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.59× ×
GPT-5.6 Sol (max) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.92× ×
GPT-5.6 Sol (max) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.47× ×
GPT-5.5 (xhigh) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.84× ×
GPT-5.5 (xhigh) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.28× ×
GPT-5.2 (high) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.70× ×
GPT-5.2 (high) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.21× ×
GPT-5.4 (xhigh) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.60× ×
GPT-5.4 (xhigh) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.12× ×
GPT-5 (high) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.53× ×
GPT-5 (high) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.08× ×
o3 (high) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.43× ×
o3 (high) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.07× ×
o1 (high) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.22× ×
o1 (high) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.06× ×
GPT-4o (gpt-4o-2024-05-13) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.29× ×
GPT-4o (gpt-4o-2024-05-13) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.03× ×
GPT-4 (gpt-4-0613) / TasteVal
Benchmark
TasteVal
Benchmark by P-Zero Research
October 2026 paper v1 · 20-model main cohort
0.22× ×
GPT-4 (gpt-4-0613) / TasteVal
Benchmark
InnovationEval
Benchmark by Epoch AI
Initial SDPO task · October 7, 2026
15%
GPT-5.6 Sol · InnovationEval
Relevant sources
GPT-6 Astra System Card ↗
system card · 2026-09-03 · available
PaperBench: Evaluating AI’s Ability to Replicate AI Research (v1) ↗
original paper · 2025-04-02 · available
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering (v1) ↗
original paper · 2024-10-09 · available
Research acceleration: The view inside OpenAI ↗
organizational research article · 2026-09-06 · available
Addendum to GPT-6 Astra System Card: GPT-6.1 Sol ↗
system card · 2026-09-29 · available
GeneBench-Pro: Evaluating Multistage Statistical Reasoning in Genomics, Quantitative Biology, and Translational Biomedicine (v4) ↗
original paper · 2026-06-30 · available
GPT-Rosalind-6 System Card Addendum ↗
system card · 2026-10-30 · available
Covert underperformance on safety research tasks ↗
organizational research article · 2026-11-04 · available