AI Inference
554 episodes · Page 4 of 24
#6211: Attention Is Still Winning: The Hybrid Model Census
Mamba layers shipped — inside hybrids. A census of Qwen, Nemotron, Kimi, Granite, and why MiniMax went back to full attention.
#6201: Inside Cloudflare's Clef: A Language Model That Can't Speak
Cloudflare's Clef reads text, images, and video but outputs only probabilities — no words. Here's what's actually under the hood.
#6198: Benchmarking a TTS Model on Your Own GPU
Real-time factor, warm-up artifacts, and why an 8-bit quantized speech model might not run any faster.
#6191: What Actually Happens During Fine-Tuning
Sixteen bytes per parameter in, two out — a walk through the forward pass, backward pass, and optimizer that turn a dataset into new weights.
#6190: When Your AI API Quietly Changes Under You
Same API name, different model behind it. How to tell when the thing you tested stops being the thing you're using.
#6185: Why AI Labs Sell a Tier Above the Flagship
Fable, Deep Think, SuperGrok Heavy — what the $300/month top tier actually does that the flagship can't.
#6184: Anthropic's Unreleased Model Two, Explained
Anthropic trained a model better than what it sells — and never released it. What that gap actually tells us.
#6173: Auditing a Sycophantic Model From the Inside
A stumped lab, a fawning model, and the diagnostic playbook nobody has a name for — from eval slicing to persona vectors.
#6170: One Endpoint, Many Boxes: Local LLM Routing
A LiteLLM-style proxy can fail over between your own GPU boxes and the cloud — but it can't see your VRAM.
#6169: What It's Like to Be a Token Inside an LLM
A token's-eye view of an LLM's forward pass — no cone, no squeezing, just a hallway where every layer scribbles on you.
#6165: Watching 20 Batch Jobs at Once
Batch APIs give you a discount, then hand you a babysitting job. Here's what actually watches a batch run — and what doesn't.
#6164: We Gave Each Host Its Own AI Agent
We split our two hosts into separate AI agents with private context windows. The bill came to 78 cents an episode.
#6162: Why Dictated Words Rewrite Themselves
Hold-to-talk dictation shows dimmed text that silently changes when you release the key. Here's the architecture behind that rewrite.
#6160: Where Do Tiny Speech Rules Live? A Ladder From Regex to LoRA
A regex notebook, a WFST grammar, or a fine-tune? Where do tiny speech-rewrite rules actually belong in a 600M-parameter model?
#6153: Decision Models: Typed Answers in One Forward Pass
Clef, Jev, and OpenAI's Decisions API return typed answers with probabilities in a single forward pass — no generated text to parse.
#6148: The Tiny Hebrew Models Behind the Show's Voice
Dicta's 300M-parameter nikudizer, Phonikud's adaptors, and the ktiv male model hiding inside a vowel adder.
#6146: Grounding Is Not Solved: DIY vs Gateway vs Native Search
Bolt-on search, gateway search, or native grounding? The plumbing didn't vanish — it just moved to someone else's bill.
#6139: Which AI Writes Our Scripts Best? We Tested Five
Five models, three scripts each, one rubric. Sol won on length, Grok on voice — and the runner-up title hides a catch.
#6135: How Many Runs Does Your AI Eval Actually Need?
Reruns shrink one kind of noise and leave another untouched — so where should your eval budget really go?
#6134: Cheap Tokens, Expensive Scripts
A model with worse list prices can cost less per script — if the cache hits. The bill hides in tokens, not rates.
#6132: How to Run a Model Eval on a Budget
Daniel is teaching himself to run evals on a real budget. Successive halving, paired comparisons, and why three clean scripts prove almost nothing.
#6127: Haiku 5.5 at 10 Cents: What the Cheap Model Actually Does
Haiku 5.5 costs 10 cents per million input tokens. We break down where it holds up, where it fails, and whether it can write this show.
#6104: Mistral AI : le pari européen de la souveraineté en intelligence artificielle
Comment Mistral AI défie la Silicon Valley avec des modèles efficaces, des partenariats stratégiques et son nouveau modèle vocal Voxtral ? Corn et Herman font le point.
#6092: 到底是谁在塑造中国的人工智能思想？
不只是实验室的老板：那些定义中国如何打造、又如何争论人工智能的研究者和思想。