KV caches: the memory behind a faster conversation
Why a language model stores keys and values, how the cache grows, and what a simple memory calculation can tell you.
Read articleEvidence, context, and a practical point of view.
Browse topics
14 articles
Why a language model stores keys and values, how the cache grows, and what a simple memory calculation can tell you.
Read articleA practical explanation of mixture-of-experts (MoE) routing, load balancing, and why sparse computation does not automatically mean lower latency.
Read articleHow draft-and-verify generation can preserve a target distribution, and why acceptance rate alone is not a speed benchmark.
Read articleWhy retrieval does not create authority, what a detector can and cannot prove, and how to design a smaller permission boundary.
Read articleA visual guide to pretraining, supervised fine-tuning, LoRA, QLoRA, RLHF, and DPO, with worked calculations and practical evaluation advice.
Read articleFollow a neuron by hand, count a small network’s parameters, calculate a gradient update, and learn how to judge generalization.
Read articleA separate visual guide to embeddings, Q/K/V, attention arithmetic, causal masks, transformer blocks, and the KV cache.
Read articleThe most useful AI project often starts with a smaller question: which part of this task is actually slowing someone down?
Read articleA clear example of retrieval-augmented generation, including where retrieval fails and when a system should say it cannot answer.
Read articleWhy a numerical representation helps search find related text, and why a high similarity score is not a fact-check.
Read articleReplace “this sounds good” with a repeatable review of correctness, evidence, completeness, and uncertainty.
Read articleA practical decision framework based on data, hardware, latency, maintenance, and the actual task—not model hype.
Read articleA repeatable way to specify the task, evidence, boundaries, and output without accumulating pages of conflicting instructions.
Read articleA practical data-minimization routine for everyday AI use, including identifiers, hidden fields, and useful substitutes.
Read articleNo articles match that search. Try a shorter phrase or clear the search.