TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts

Table 2: Opus 5.5 (max); Experimental compute multiplier

Section 3, printed page 7: refusal is missing data; common-seven-task sensitivity is separate

Table 2: Opus 5.5 (max); Normalized final performance multiplier

Section 3, printed page 7: refusal is missing data; common-seven-task sensitivity is separate

Table 2: Fable 5.1 (max); Experimental compute multiplier

Section 3, printed page 7: refusal is missing data; common-seven-task sensitivity is separate

Table 2: Fable 5.1 (max); Normalized final performance multiplier

Section 3, printed page 7: refusal is missing data; common-seven-task sensitivity is separate

Table 2: GPT-6 Astra (max); Experimental compute multiplier

Table 2: GPT-6 Astra (max); Normalized final performance multiplier

Table 2: Opus 5.0 (max); Experimental compute multiplier

Table 2: Opus 5.0 (max); Normalized final performance multiplier

Table 2: Opus 4.8 (max); Experimental compute multiplier

Table 2: Opus 4.8 (max); Normalized final performance multiplier

Table 2: GLM 5.3 (max); Experimental compute multiplier

Table 2: GLM 5.3 (max); Normalized final performance multiplier

Table 2: GPT-5.6 Sol (max); Experimental compute multiplier

Table 2: GPT-5.6 Sol (max); Normalized final performance multiplier

Table 2: Opus 4.7 (max); Experimental compute multiplier

Table 2: Opus 4.7 (max); Normalized final performance multiplier

Table 2: GPT-5.5 (xhigh); Experimental compute multiplier

Table 2: GPT-5.5 (xhigh); Normalized final performance multiplier

Table 2: Kimi K3 (reasoning on); Experimental compute multiplier

Table 2: Kimi K3 (reasoning on); Normalized final performance multiplier

Table 2: Opus 4.6 (max); Experimental compute multiplier

Table 2: Opus 4.6 (max); Normalized final performance multiplier

Table 2: GPT-5.2 (high); Experimental compute multiplier

Table 2: GPT-5.2 (high); Normalized final performance multiplier

Table 2: GPT-5.4 (xhigh); Experimental compute multiplier

Table 2: GPT-5.4 (xhigh); Normalized final performance multiplier

Table 2: Opus 4.5 (24k thinking budget); Experimental compute multiplier

Table 2: Opus 4.5 (24k thinking budget); Normalized final performance multiplier

Table 2: GPT-5 (high); Experimental compute multiplier

Table 2: GPT-5 (high); Normalized final performance multiplier

Table 2: Kimi K2.5 (max, 96k output); Experimental compute multiplier

Table 2: Kimi K2.5 (max, 96k output); Normalized final performance multiplier

Table 2: o3 (high); Experimental compute multiplier

Table 2: o3 (high); Normalized final performance multiplier

Table 2: o1 (high); Experimental compute multiplier

Table 2: o1 (high); Normalized final performance multiplier

Table 2: GPT-4o (gpt-4o-2024-05-13); Experimental compute multiplier

Table 2: GPT-4o (gpt-4o-2024-05-13); Normalized final performance multiplier

Table 2: GPT-4 (gpt-4-0613); Experimental compute multiplier

Table 2: GPT-4 (gpt-4-0613); Normalized final performance multiplier

Sections 2–4; Figures 1, 4; Table 2; Appendix H.1–H.3

Primary publication source