Level 5: AI / LLM systems · L5.1 LLM fundamentals for architects
Latency and cost of a model call
Explain prefill vs decode, TTFT vs TPOT and token pricing.
12 min · Pro
What you take away
- One call is two different machines: prefill reads the prompt in parallel and is compute-bound; decode emits one token at a time and is memory-bandwidth-bound.
And 3 more inside the class, in Pro.
How the class is laid out
- The problem
- Two machines in one call
- Why batching changes the economics
- Check your understanding