Level 5: AI / LLM systems · L5.1 LLM fundamentals for architects
Transformers for architects
Explain attention and context windows at the level architects need.
15 min · Pro
What you take away
- Attention lets every token look at every other token, which is what makes the model good and what makes long prompts expensive.
And 3 more inside the class, in Pro.
How the class is laid out
- The problem
- What attention actually costs
- The context window is working memory
- What you do not need
- Check your understanding