Model Evaluation Cheat Sheets

Ablation Study cheat sheet

A removal proves credit only against a fixed baseline, one change per variant, and more than one seed.

#Model EvaluationAnimated

→ Topic: Ablation Study

Benchmark Contamination cheat sheet

A score proves capability only while the test stayed unseen; once questions leak into training, it measures memory.

#Model EvaluationAnimated

→ Topic: Benchmark Contamination

Confusion Matrix cheat sheet

Every precision, recall or accuracy figure is arithmetic on four cells; read them per class before trusting the total.

#Model EvaluationAnimated

→ Topic: Confusion Matrix

ELO Rating for LLMs cheat sheet

An arena rank shows which answers people preferred; inside the confidence interval, a rank change is a tie.

#Model EvaluationAnimated

→ Topic: ELO Rating for LLMs

Eval harness: reading AI scores cheat sheet

An eval harness keeps the questions, the prompt and the grading fixed so scores compare. Follow one real question through it and see one model score 0 % to 63 %.

#Model Evaluation

→ Read the article

Evaluation Harness cheat sheet

Hold the tests and the scoring fixed, swap only the model, and the scores become comparable.

#Model EvaluationAnimated

→ Topic: Evaluation Harness

Human Evaluation for AI cheat sheet

A score counts only after raters agree on the rubric; low agreement sends you back to rewrite it, not to hire raters.

#Model EvaluationAnimated

→ Topic: Human Evaluation for AI

LLM-as-a-Judge cheat sheet

A judge model scales grading, but its verdicts earn trust only after a precise rubric is checked against human labels.

#Model EvaluationAnimated

→ Topic: LLM-as-a-Judge

MMLU Benchmark cheat sheet

A high score shows what a model absorbed in training, not how it handles your own task.

#Model EvaluationAnimated

→ Topic: MMLU Benchmark

Model Evaluation cheat sheet

No benchmark, metric or leaderboard settles fit alone; decide the metric first and test on your own tasks.

#Model EvaluationAnimated

→ Topic: Model Evaluation

Precision Recall and F1 Score cheat sheet

Raising the threshold buys precision with missed positives; decide which error costs more before you tune.

#Model EvaluationAnimated

→ Topic: Precision Recall and F1 Score

SWE-bench cheat sheet

A pass on hidden tests scores a curated set of open-source issues, not your own codebase.

#Model EvaluationAnimated

→ Topic: SWE-bench

What is an ablation study? cheat sheet

An ablation study removes or replaces one part of a machine-learning system and measures what changes.

#Model Evaluation

→ Read the article