AI4AI-Bench

Results by version

Benchmark version

Metric

Results to compare

Protocol

Paper v1

2026-08-20 · 10 tasks

Ten algorithm families; fixed hidden evaluator retrains submissions.

Metric definitions

Mean normalized score (normalized score): 0.1 is repository baseline; 1 is task optimum.

AI4AI-Bench · Mean normalized score

limited comparison

Different effort grids and costs; no automatic delta.

Published results · Paper v1
Key	System / organization	Metric	Reported result	Date	Protocol	Verification	Evidence
3	Claude Opus 5 / Claude Code effort aggregate	Mean normalized score	0.250 normalized score	2026-08-20	a4-opus-p	benchmark author reported	AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement ↗
4	GPT-5.6 Sol / Codex effort aggregate	Mean normalized score	0.191 normalized score	2026-08-20	a4-sol-p	benchmark author reported	AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement ↗
1	Kimi K3 / Claude Code effort aggregate	Mean normalized score	0.174 normalized score	2026-08-20	a4-kimi-p	benchmark author reported	AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement ↗
5	Claude Sonnet 5 / Claude Code effort aggregate	Mean normalized score	0.145 normalized score	2026-08-20	a4-sonnet-p	benchmark author reported	AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement ↗
6	GPT-5.6 Terra / Codex effort aggregate	Mean normalized score	0.135 normalized score	2026-08-20	a4-terra-p	benchmark author reported	AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement ↗
2	GPT-5.6 Luna / Codex effort aggregate	Mean normalized score	0.117 normalized score	2026-08-20	a4-luna-p	benchmark author reported	AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement ↗
—	Claude Opus 5 / Claude Code, medium effort	Mean normalized score	0.288 normalized score	2026-08-20	a4-opus-medium-p	benchmark author reported	AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement ↗

Version lineage

Paper v1

Reference points

normalization anchor

0.1 · normalized score

The original repository algorithm maps to 0.1; this is not a human-researcher threshold.

What this reference means: unknown · Where it applies: established

AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement ↗

Availability

What is publicly available
Resource	Status
public description	yes
public results	yes
public tasks	yes
public code	yes
public evaluation service	unknown

Official sources

AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-ImprovementSections 2–3; Figure 2 and Table 2Original source ↗