LLM Inference & Large-Scale Deployment — Study Guide

An illustrated, exam-aligned study guide for serving large language models in production: how inference differs from training, the prefill/decode split and the KV-cache, batching (static/dynamic/continuous), PagedAttention and FlashAttention, quantization (int8/int4, GPTQ/AWQ/GGUF), speculative decoding, tensor parallelism, GPU-memory math, latency metrics (TTFT/TPOT), serving stacks, caching, autoscaling, cost, observability, and guardrails. Diagrams, comparison tables, and worked GPU-memory examples throughout.

Key facts

• Course price: $195 (one-time payment)
• Field: Artificial Intelligence & LLMs
• Prepares for: LLM Inference & Large-Scale Deployment
• 5 modules · 16 lessons · about 3 hours of lessons
• Self-paced, online, illustrated lessons

Course outline

1. From Training to Inference — Why serving is a different problem: the forward-only decode loop, the prefill/decode split, and the KV-cache that defines LLM inference.
   • Inference Is Not Training
   • The Autoregressive Decode Loop
   • Prefill vs Decode: Two Phases, Two Bottlenecks
2. The KV-Cache & GPU-Memory Math — Why the KV-cache exists, what it costs, and how to size weights + KV + activations for a real deployment.
   • Why the KV-Cache Exists
   • KV-Cache Memory: The Formula
   • Sizing a Deployment: Weights + KV + Activations
3. Batching & Attention Kernels — Static, dynamic, and continuous batching; PagedAttention's paged KV; and FlashAttention's IO-aware exact attention.
   • Static, Dynamic & Continuous Batching
   • PagedAttention: Paging the KV-Cache
   • FlashAttention: Exact, IO-Aware Attention
4. Quantization & Faster Decoding — Int8/int4 weight and activation quantization (GPTQ, AWQ, GGUF), the accuracy tradeoff, and speculative decoding.
   • Quantization: Methods & Tradeoffs
   • The Memory-Bandwidth View of Decode
   • Speculative & Assisted Decoding
5. Serving at Scale — Latency metrics, throughput/latency tradeoffs, tensor parallelism, serving stacks, caching, autoscaling, cost, observability, and guardrails.
   • Latency Metrics & the Throughput Tradeoff
   • Scaling the Model: Tensor Parallelism & Stacks
   • Caching, Autoscaling & Cost
   • Observability & Serving-Time Guardrails