Description

Every time you send a prompt to a chatbot, a GPU somewhere has to load billions of weights, juggle hundreds of other users and stream your answer back token by token, fast enough to feel instant and cheap enough to make business sense. This course explains how that actually works.

LLM Inference in 100 Questions is a complete, visual tour of large language model serving and optimization, organized as 100 short, focused lessons of about 3 minutes each. Every lesson answers one concrete question, with animated diagrams, worked numbers and a clear takeaway.

What you will cover

• How a model generates an answer: tokens, attention, prefill vs. decode, the KV cache and streaming

• Hardware and bottlenecks: GPU spec sheets, HBM vs. SRAM, NVLink, memory sizing, arithmetic intensity and the roofline model

• Measuring and costing: TTFT, ITL/TPOT, throughput, Little's law, benchmarking and the cost of a million tokens

• Batching: static, dynamic and continuous batching, chunked prefill, schedulers and preemption

• Attention and memory: MQA, GQA, MLA, FlashAttention, PagedAttention and KV cache quantization

• Shrinking the model: FP16, BF16, INT8, FP8, FP4, GPTQ, AWQ, distillation and sparsity

• Never doing the same work twice: prefix caching, RadixAttention, semantic caching and speculative decoding (Medusa, EAGLE)

• Serving architecture: load balancing, autoscaling, tensor/pipeline/expert parallelism, disaggregated prefill and decode, cache-aware routing

• Engines and platforms: vLLM, SGLang, TensorRT-LLM, llama.cpp, managed cloud options, build vs. rent

• What's next: multimodal serving, multi-LoRA, edge inference, semantic routing, RL and new accelerators

The course is based on the book "100 Soruda Büyük Dil Modelleri" (Large Language Models in 100 Questions) by Buğra Ayan. The lessons are narrated by the author. English subtitles are included.

No GPU is required to follow along. By the end, you will be able to reason about latency, throughput and cost like an inference engineer, and choose the right optimizations for your own workload.