Topic:Inference
All essays filed under "Inference".
What LLMs Do at Inference: A Deep Dive Under the Hood
Updated:A step-by-step, reference-backed explanation of what happens during LLM inference: tokenization, embeddings, prefill & decode phases, KV caching, decoding strategies, bottlenecks and optimizations like quantization, FlashAttention and speculative decoding.
KV Cache Explained - A Deep Dive into Transformer Optimization
Updated:A Deep Dive into Transformer Optimization
Top-k vs. Nucleus Sampling - Decoding the Secrets of AI Text Generation
Updated:Decoding the Secrets of AI Text Generation
GPU vs TPU - Decoding the Battle of AI Accelerators in 2025
Updated:Decoding the Battle of AI Accelerators in 2025