flipbudget — MATH-Hard shadow leaderboard
Which of 378 real pairwise model comparisons on MATH-Hard remain
identified once scorer misclassification is accounted for.
Data + code: github.com/Rajveer-code/flipbudget
all pairs
not identified only
degenerate only
Higher-scoring model
Lower-scoring model
Raw gap
Corrected gap
Flip budget
Identified?
No pairs match this filter.
Real data, not illustrative. Generated from
e1_case_b_TRUE.json
and
e5_leaderboard.json .
"Corrected gap" flips sign relative to "raw gap" exactly where a pair is
marked not identified. "Flip budget" is the least differential scorer
error that would put the pair's ranking in doubt (T2); "—" means the pair
was already inconclusive under ordinary sampling noise alone (degenerate).