Cianplan

Level 4: Reliability, security & cloud · L4.2 Observability

Observing LLM apps

Trace prompts, tokens, cost and latency per LLM call.

11 min · Pro

What you take away

  • An LLM call is an ordinary span with extra attributes: model, prompt version, input and output tokens, finish reason, cache hit.

And 3 more inside the class, in Pro.

How the class is laid out

  1. The problem
  2. The LLM span
  3. Two latencies, and one of them is a parameter you set
  4. Cost lives per request, not per call
  5. The 200 that is wrong
  6. Three signals
  7. Golden signals, RED, USE
  8. Tracing
  9. LLM apps
  10. Check your understanding