Transformer Architecture
226 episodes · Page 2 of 10
#6178: Why Your Voice Clone Sounds American
Your cloned voice drifts toward American English. Here's what's actually happening — and why "neutral" is a myth.
#6174: What Your Voice Reveals: Accent, Distance, and Movement in Audio AI
What can AI actually hear in your voice? Accent detection, speaker distance, whale songs, and the gap between deployed audio models and chatty gene...
#6169: What It's Like to Be a Token Inside an LLM
A token's-eye view of an LLM's forward pass — no cone, no squeezing, just a hallway where every layer scribbles on you.
#6148: The Tiny Hebrew Models Behind the Show's Voice
Dicta's 300M-parameter nikudizer, Phonikud's adaptors, and the ktiv male model hiding inside a vowel adder.
#6110: The Chatbot OpenAI Almost Shelved
Instruction models existed for months. Then a rumour about Anthropic pushed OpenAI to ship a chat box in two weeks.
#6105: Не только Китай: ИИ в России, Индии и Японии
Китай занимает почти всё внимание в разговорах об ИИ, но Россия, Индия и Япония тоже создают мощные региональные модели с необычной архитектурой.
#6072: China Beyond Beijing and Shanghai: A Regional Guide
Most Westerners know two Chinese cities. Here's the rest of the map — regions, megacities you've never heard of, and which ones you can actually re...
#6070: The Papers That Moved AI: From AlexNet to This Year's Fights
The classics all said 'learn it from data and scale it up'. This year's most-discussed AI papers are auditing what scaling took for granted.
#6052: What's Actually in the Hugging Face ASR Catalog?
Four real architectural bets sit underneath thousands of Hugging Face ASR repos. Here's how to tell the bets from the repackagings.
#6029: One Model, Two Modes: Does AI Consolidation Hold Up?
OpenAI hides a router, DeepSeek merged two modes, Anthropic removed the knob. Is the model landscape really consolidating?
#6027: Can You Snap Two Fine-Tunes Together Like LEGO?
Two fine-tunes, one base model, and a question from Jerusalem: can you merge Hebrew code-switching with technical vocab — without retraining?
#6024: What's Actually Inside the KV Cache?
890 bytes per token: why the KV cache is both what makes LLMs practical and what makes long context expensive.
#6021: Why Chat Templates Break Models Silently
Every chat API flattens your messages into special tokens. Get one delimiter wrong and the model just gets quietly dumber.
#6016: What 70 Billion Parameters Actually Means
A single artificial neuron, built from scratch — then the hard part: does that neuron analogy still describe a real transformer?
#6014: Why Every AI Model Is Secretly a Decoder
Encoder, decoder, or both? The three transformer shapes, why one won, and why BERT still refuses to die.
#6013: What Happens Inside One LLM Forward Pass
One trip through the network, layer by layer — and why a classifier needs one pass while an LLM needs hundreds.
#6012: Inside the Decode Loop: How LLMs Generate One Token at a Time
The model doesn't choose tokens or decide when to stop — the engine does. A step-by-step tour of one decode iteration.
#6011: Inside LLM Inference: Prefill, Decode, and the KV Cache
One 8K-token sequence on a 13B model eats 6.7GB of KV cache—while the weights are only 26GB. Here's what actually happens per request.
#5975: Merging vs Distillation: Can Two Models Become One?
Merging and distillation get lumped together, but they're opposites. One composes existing capability; the other has to teach it.
#5974: One Model or Two? ASR Cleanup Explained
If speech models can punctuate and capitalize on their own, why does a separate 484MB cleanup model exist at all?
#5945: How a Reward Model Actually Trains an LLM
One number, no gradients, four models in the room. What a reward model actually does inside the RLHF loop.
#5899: Same GPU, Same Weights, Different Answers
Identical hardware, identical weights, different engine — and answers that shift by up to 16.6 points.
#5796: 900 Tonnes on Public Roads: The Weirdest Cargo Moves
544 wheels, 66 hours on the road with three elephants, and a spaceship on Crenshaw Boulevard — the strange industry that moves the impossible.
#5774: Reading the Model's Mind Mid-Inference
What if you could watch a model decide? Inside the tools that open up inference mid-computation — and the limits they hit.