Back to Study Plans

30 Problems

Inference Engineering

Implement attention variants, sparse experts, decoding, quantization, serving schedulers, and distributed inference systems from first principles.

5 Easy13 Medium12 Hard

Unlock the full study plan

Upgrade to Pro to access the full curriculum.

6 sections

Implement Scaled Dot-Product Attention Implement Multi-Head Attention (MHA)Implement Multi-Query Attention (MQA)Implement Grouped-Query Attention (GQA)Implement Multi-Head Latent Attention (MLA)Attention Architectures Quiz10 questions

Implement Rotary Position Embeddings for Decoding Implement FlashAttention with Online Softmax Implement Temperature, Top-k, and Top-p Sampling Implement Sparse MoE Top-k Expert Routing Implement MoE Token Dispatch and Expert Aggregation Efficient Transformer Components Quiz10 questions

Implement Autoregressive Decoding with a KV Cache Calculate KV Cache Memory for MHA, MQA, GQA, and MLA Implement PagedAttention Block Allocation Implement Prefix Cache Matching and Reuse Implement Speculative Decoding and Token Verification KV Cache and Decoding Quiz10 questions

Implement Symmetric INT8 Quantization Implement Per-Channel Weight Quantization Implement Group-Wise INT4 Quantization Implement Block-Wise FP8 Quantization Analyze Prefill and Decode with the Roofline Model Quantization and GPU Performance Quiz10 questions

Calculate P50, P95, and P99 Inference Latency Compute TTFT, TPOT, ITL, and Token Throughput Implement Dynamic Request Batching Simulate Continuous Batching Simulate Chunked Prefill Scheduling Batching and Serving Metrics Quiz10 questions

Estimate Model Memory and Minimum GPU Count Calculate Tensor Parallel Communication Cost Implement Expert Parallel Routing Across GPUs Simulate Disaggregated Prefill and Decode Serving Optimize Throughput and Cost Under a Latency SLO Distributed Inference Systems Quiz10 questions

Summary

Total30

Easy5

Medium13

Hard12

Progress0%

Quizzes0/6