Series · Builders · Step 5
LLM Infrastructure & Serving
How do large language models actually run — and where does the money go? This series walks through the inference engine itself: the difference between the prefill and decode phases, the role of the KV cache, the numerical compression of weights, the batched scheduling of requests, and what it takes to serve a large model on a single GPU. The goal is a clear, vendor-neutral picture of where cost and speed are really decided — foundational knowledge that serves you whatever your stack.
6 pieces, in order
-
Part 1: PagedAttention and continuous batching: how one server answers more users
Step 5 · 5 min -
Part 2: How LLM inference actually works: prefill vs decode
Step 5 · 4 min -
Part 3: What quantization actually does: precision loss and vector-space collapse
Step 5 · 5 min -
Part 4: MLP is the model's memory: where knowledge lives
Step 5 · 4 min -
Part 5: A 70B model on one GPU: a practical guide to quantization
Step 5 · 4 min -
Part 6: Where LLM serving costs actually go
Step 5 · 4 min