Cianplan

Level 5: AI / LLM systems · L5.6 Evals & guardrails

LLM-as-judge done right

Build and validate a judge against human labels.

12 min · Pro

What you take away

  • Reach for a judge only after assertions and deterministic metrics have failed to cover the criterion — they are cheaper, faster and exactly repeatable.

And 3 more inside the class, in Pro.

How the class is laid out

  1. The problem
  2. A new examiner marks the same papers first
  3. Reach for the cheapest check that works
  4. Check your understanding