A: Single-turn QA is "input → one output → compare to a reference," one-shot and near-deterministic; agent eval measures a multi-step trajectory whose success criterion is the terminal state satisfying the goal (execution-based: run unit tests / check environment state), not string matching. Looking only at accuracy misses three agent-specific things: reliability across runs (pass^k), trajectory quality (any reward hacking / lucky guess), and score decay over time (contamination + saturation).

Follow-up: Why do agent benchmarks prefer execution-based criteria over answer matching? → Because the agent's "correct" is often not a unique string (code edits have many forms, web operations have many paths); it can only be judged by "is the final executable goal state reached"; execution-based is also harder to cheat (if the criterion is hidden).

A: Contamination = test samples (problems, even official solutions) enter the training corpus, so the model "remembers" rather than "solves," and the score measures memory. It is especially severe for agent benchmarks because many tasks originate from public GitHub (SWE-bench's issues + fix PRs are public and crawled), so problems and gold solutions are naturally in the training set; add frontier models repeatedly farming these boards and overfitting + memorization compound.

Follow-up: How do you detect that a model is contaminated on a benchmark? → ① n-gram overlap scan of the training corpus; ② give only the statement, no environment, and see if it reproduces the official fix from memory; ③ compare score cliffs on structurally identical problems before vs after the training cutoff.

A: pass@k = at least one of k attempts succeeds (capability upper bound, optimistic); pass^k = all k succeed (reliability, see foundations §9). Neither is the truth — both are estimators: in practice you draw $n$ samples per problem to estimate, and small $n$ means high variance. pass@k has the unbiased estimator $1-\binom{n-c}{k}/\binom{n}{k}$ ($c$ = number correct among $n$), lower-variance than "count one hit/miss."

Follow-up: Why must reporting pass@k also report $k$, temperature, and $n$? → pass@k's value varies with $k$ and the sampling distribution (temperature) — temperature changes per-sample success rate and diversity; whereas $n$ (the number of samples used to estimate it) only sets the variance / confidence interval and does NOT change pass@k's expected value (unless you actually change $k$ or do best-of-$n$). So missing $k$ / temperature → not comparable; missing $n$ / seed → not reproducible.

A: Two lines compound. Contamination line: OpenAI's elicitation experiments show frontier models on some tasks reproduce the official fix from memory given only the statement → scores reflect "how much benchmark was seen" rather than real ability. Saturation/defect line: OpenAI audited the 138 problems its o3 did not consistently solve across 64 runs (27.6% of 500) and found 59.4% have substantial test/description defects — 35.5% (≈49) too narrow (reject functionally correct solutions), 18.8% (≈26) too wide (demand functionality the statement never mentioned), the rest ~5% description defects; i.e. part of the "failures" are benchmark misjudgments. Conclusion: gains on Verified no longer reflect real development ability. OpenAI pivoted to the contamination-resistant SWE-bench Pro (same cohort ~70–80%→~23–58% magnitude).

Follow-up: What to watch when citing "drops to ~23–58% on SWE-bench Pro"? → That is a single 2025–26 source (Scale leaderboard), a shifting concrete score; use it only to illustrate the "magnitude gap from saturation/contamination," not as a stable ranking or fixed capability value.

A: It says a benchmark compares model × harness, not pure model — prompt template, toolset, parser, step budget, retrieval strategy all materially move the score (SWE-bench's early gains were largely scaffold engineering). A responsible report must carry: harness version + toolset + max steps + temperature + seed + sample count $n$ + date. A lone number missing these is incomparable and irreproducible.

Follow-up: What does this mean for "reading others' technical reports"? → Seeing "we reached X%" without the harness and date, discount comparability; before comparing scores across reports, first confirm same harness, same benchmark version, whether contamination-resistant.

A: It evaluates only on contest problems newly published after the model's training cutoff — since the problems appeared after training, they cannot have been seen during training, cutting contamination mechanistically; "living" = continuously roll in new problems and retire old ones, staying fresh long-term. Limits: ① problem types skew algorithmic contests, narrower than real software engineering (no repo-level issues); ② requires ongoing operational effort; ③ different models have different cutoffs, so strict comparability requires windowing per each cutoff.

Follow-up: Time-window anti-contamination vs "private hold-out / expert authoring" (GDPval, SWE-bench Pro private) — what suits each? → Time windows suit domains with a steady stream of new problems (contests); private / expert authoring suits domains with no natural new-problem stream, or that need closeness to real professional tasks, at the cost of human effort for authoring + grading.

A: Filter by three matching criteria in order — ① Action-space match: what is the agent's operation granularity? Fixing code → SWE-bench Pro / LiveCodeBench; operating websites → WebArena; controlling an OS GUI → OSWorld; multi-turn tool + user interaction → τ-bench; ML engineering → MLE-bench; general-purpose QA + tool use → GAIA. ② Judging-method reliability: can you tolerate LLM-judge bias? High-stakes scenarios (code deployment, financial transactions) → prefer execution-based + hidden-criterion benchmarks; low-stakes exploratory scenarios → LLM-judge acceptable for initial filtering. ③ Maintenance status: is it actively maintained / living? A saturated benchmark (SWE-bench Verified, retired) cannot be the sole basis; living benchmarks (LiveCodeBench) or recently released, community-active ones are more trustworthy.

Concrete mapping:

Follow-up: What if no existing benchmark fits your scenario? → ① Check whether a general-purpose benchmark (GAIA / τ-bench) can serve for an initial screen; ② if not, build a domain-specific eval: the key is making the success criterion executable (script / unit test / terminal-state check) and keeping it invisible to the agent; ③ estimate the construction cost: typically dozens to hundreds of items depending on baseline success rate + effect size + desired CI width (do a power analysis to determine item count; as a rough heuristic often ≥100), plus ongoing maintenance (item rotation to prevent overfitting).

A: LLM-as-judge for agent trajectories carries three categories of bias risk that have been repeatedly documented in the literature:

1. Position / order bias: the order in which answers appear in the prompt, and whether content sits near the prompt's end, significantly affects judge scores. Gets worse with longer trajectories — agent multi-step trajectories are inherently longer than single-turn QA, so the last steps dominate the judge far more than the first.
2. Self-preference: a judge from the same model family tends to give higher scores to same-family agents; this has been observed across GPT, Gemini, and Claude families in multiple studies — directionally consistent, though the magnitude varies by model/task rather than being a fixed value.
3. Style / verbosity / authority bias: if the agent's trajectory includes confident, detailed, or authoritative-sounding explanation text, the LLM judge is more inclined to mark it correct even when the underlying action is wrong — the judge is influenced by rhetoric rather than facts (sometimes called verbosity bias, authority bias, or rationale bias).

When LLM-judge is usable:

• Relative ranking (A vs B which trajectory is better) — more reliable than absolute scoring; biases partially cancel in pairwise comparison.
• Coarse filtering — screen out obviously bad trajectories (didn't call a tool / infinite loop / empty response), with rule-based criteria as the primary gate and LLM as the fallback.
• Assistive pre-labeling — pre-annotate before human review to accelerate manual checking.

When it is NOT usable / requires extreme caution:

• As the reward source for RL training — biases will be actively searched and amplified by the policy, turning the objective into "please the judge" rather than "complete the task."
• As an absolute pass/fail criterion — reliability insufficient for a high-stakes single decision.
• Cross-model-family capability ranking — self-preference makes the ranking systematically incomparable.

Follow-up: Is there a middle ground more reliable than LLM-judge but more flexible than execution-based? → Programmatic rules + LLM combinations: dimensions that can be ruled (step budget, forbidden-tool calls, action-format validity) use deterministic rules; dimensions that need "semantic judgment" (is this step directionally correct) use LLMs, but only in a statistical sense (mean + variance across multiple judges), never relying on any single judge for a single decision.

A: Synthesizing §3–§5: ① anti-contamination — use problems after the training cutoff (time window) or a private hold-out, and audit contamination (memorization-recall test); ② anti-reward-hacking — use hidden extra unit tests + environment terminal-state checks (invisible to the agent) as the success criterion, preventing test-editing / empty-impl-passes-CI; audit the unit-test quality itself (avoid too-weak→false-positive, too-strict→false-negative); ③ reproducibility — fixed seed, public harness, report the unbiased pass@k estimate over $n$ samples + interval + version date; ④ reliability — report pass^k not just pass@1, exposing multi-run stability; ⑤ living — rotate problems periodically to prevent long-term overfitting.

Follow-up: Why is "hidden criteria" the key to anti-hacking? → Once the agent can see the criterion (assertions / unit tests), it satisfies the criterion rather than truly solving (edit the test file, hard-code the expected output); hiding the criterion where the agent can't see it forces it to actually reach the goal state.

A: outcome-only blind spot — misses "lucky guess / lucky path" and reward hacking, and does not diagnose which step failed or give partial credit; trajectory blind spot — "who judges each step" is hard, rules miss coverage, LLM-judge carries position/self-preference. execution-based verifier false-negative/false-positive: tests too weak → a wrong solution sneaks through (false positive), too strict → a functionally correct solution is rejected (false negative, i.e. SWE-bench Verified's defective items). Mitigations: ① audit + strengthen the test suite (diverse inputs, hidden extra assertions); ② cross multiple criteria (unit tests + terminal-state check + human review of a subset when needed); ③ keep human-in-the-loop review for high-risk items; ④ handle verifier noise robustly in RL (don't amplify a single weak signal as truth).

Follow-up: Why is a weak verifier more dangerous in RL training than in static eval? → In static eval a weak verifier just mismeasures; in RL it is the reward source, and reward hacking is actively searched and amplified by the policy — the agent optimizes toward "fooling the weak test" rather than "truly solving," getting worse as it trains.

A: Carry the precise scope. That "multi-agent orchestrator executes arbitrary malicious code" number (a study reports 58–90%, up to 100% in individual configs) is "the attack success rate under a specific web injection attack," i.e. the success rate of a control-flow hijack, not "agents misbehave at this rate in normal deployment." Sabotage is similar: Anthropic's sabotage evaluations are an evaluation framework; cite that it is a framework / specific setting, and if a circulated percentage can't be traced to the original setting, drop it. Principle: safety numbers are almost always bound to a threat model + experimental setting; citing one detached from the setting distorts it.

Follow-up: Why are safety-eval numbers more easily misquoted than capability-eval numbers? → Safety numbers are often upper bounds under "worst case / a specific attack," naturally eye-catching and easy to quote out of context; and they are extremely sensitive to the threat model (change the attack surface and the conclusion changes), so quoting one detached from the setting almost inevitably inflates it.

A: The standard pipeline is benchmark eval → reward design → RL training → re-eval. Every step can fail:

1. Same benchmark for both eval and reward: if the same verifier / test suite serves as both the RL reward and the final evaluation, the agent learns to "game this specific verifier" rather than truly solve. You must separate eval and train verifiers — the criterion used during training and the one used for final evaluation are two independent sets.
2. Weak verifier amplified in RL (echoing Q10): in static eval a weak verifier merely "mismeasures"; in RL it is the reward source, and reward hacking is actively searched and amplified by the policy — the agent optimizes toward "fool the weak test" (edit the test file, empty impl passes CI), getting worse as it trains.
3. Distribution shift: RL changes the policy → trajectory distribution changes → a verifier calibrated on pre-RL trajectories may no longer be valid post-RL — the agent enters behavior regions the verifier has never seen, and the false-positive/false-negative rate becomes unknown.
4. Proxy reward gap: the benchmark measures "task success," but the RL reward is a proxy for task success (unit tests passing / terminal-state check passing / LLM-judge score). The gap "success ↔ proxy" is where reward hacking thrives — the agent optimizes the proxy, not the true objective.
5. RL runs contaminate the benchmark: if RL training rounds use benchmark problems for environment interaction (not just final eval), those problems cannot be reused — the agent has already trained on them; testing them again is contamination.

Remedies: ① physically isolate eval and training problem sets (independently sampled, not a random split — agent-eval problems are naturally few, random splits are not safe enough); ② use training-dedicated hidden criteria for RL reward (verifiers independent of final eval, with zero overlap); final-eval criteria must be at least equally strong and fully held-out; ③ periodically spot-check agent trajectories manually for reward-hacking signals; ④ use multiple heterogeneous reward sources for cross-validation rather than relying on a single signal.

Follow-up: Are eval/train verifier split and eval/train problem split the same thing? → No. The problem split is "which problems are for training vs evaluation." The verifier split is using different criteria — during training expose only a training-dedicated weak criterion (few visible unit tests), and during evaluation use a held-out full-strength criterion set (zero overlap with training criteria). Both layers are needed; one layer alone is insufficient.

A: Academic benchmark scores measure capability ceiling under ideal conditions. A production agent must also be assessed along five additional dimensions:

Follow-up: How is shadow deployment done concretely, and what do you watch? → Deploy the agent in an isolated environment with the same distribution as production (production traffic mirroring / production snapshot replay / dry-run mode), with writes routed through no-op / sandbox adapters so there are no real side effects (no payments triggered, no external messages sent, not visible to real users); run silently for days to weeks, tracking:

• Shadow metrics (measurable offline): real-distribution task success rate vs academic benchmark gap, per-task cost distribution, safety red-line trigger rate, latency/step distribution
• Live metrics (require canary / A-B / assisted-human deployment to measure): user abandonment rate, human-handoff rate, satisfaction — these are NOT measurable in shadow mode and must be called out separately

The gap between these numbers and the academic benchmark scores is the chasm between "can solve problems" and "can do the job." Before shadow deployment, confirm: privacy/PII compliance (whether mirrored traffic contains sensitive data), write-operation isolation completeness, and that an approval process is in place.