AI Inference
554 episodes · Page 8 of 24
#5430: Two Boxes: ASR and the Text Fixer Behind It
Punctuation, casing, ITN, disfluency — the four-job layer between raw ASR output and text you can actually read.
#5420: How Keyboard Middleware Actually Works
A keyboard isn't one thing — it's a stack. Here's how input methods, composition buffers, and hooks really work on Linux, Android, and Windows.
#5419: The Attention Budget in Your Pocket
Why phone dictation runs out of room, and how bounded attention windows buy you punctuation without blowing your memory budget.
#5411: Fine-Tuning at 4-Bit vs 16-Bit: What It Really Costs
QLoRA cuts fine-tuning VRAM 15x and cost up to 85% — but you pay in training time, quality, and safety alignment.
#5409: What Makes a Model "Agentic"? Atria Dawn Preview
A 744B-parameter model drops quietly on Hugging Face. Is "agentic" a real model category, or just a deployment pattern?
#5408: When Small NLP Models Beat the LLM
Feature extraction, fill-mask, token classification — the classic NLP tasks still have a job. Here's when a small model beats a frontier API.
#5397: Chaining Small Models for Voice Cleanup
Six cleanup stages at 97% accuracy each compound to 83% end-to-end. So how many small models can you actually chain?
#5391: The Software Behind Urgent Care Triage
Big buttons, emoji vitals, and a system that says "order IV" — what's actually running behind the triage screen?
#5390: Chaining Small Models for Dictation Cleanup
Daniel's Android dictation fork won't render "three point five" as a decimal. How many models does cleanup actually need?
#5384: Android ASR Runtimes: LiteRT, ExecuTorch, and Why Your Phone Has No VRAM
Why does your phone have no VRAM number? A tour of Android's runtime layer and what it takes to run ASR locally.
#5345: Forecasting for Cranes, Fires, and 15-Minute Decisions
The public five-day forecast is a loss leader. Here's what bespoke weather looks like when money and lives ride on it.
#5327: Catching Duplicate Submissions Before They Burn Credits
A dropped webhook response caused a double submission. Here's how to build a screener that catches duplicates before generation runs.
#5301: Why Talking Robots Are Really a Committee
That humanoid chatting while it moves? It's not one brain — it's a stack of separate models glued together.
#5285: Two Queues, One Cron Tick: Serverless Batch Pipelines
What does a queue look like when your process is disposable? Two queues, one cron tick, and a 50% batch discount.
#5230: Why Your Research Agent Won't Stop Searching
Most agent search runs add no new evidence — and 66-85% of the context window is just search snippets. Here's what actually stops the loop.
#5188: DeepSeek's Point Release That Isn't
DeepSeek shipped a whole new architecture and called it a point release. Here's what actually changed inside the model.
#5184: AI Is a Number Factory, Not a Wordsmith
Under the prose, every AI model is just matrices of floating-point math. So where does the randomness actually come from?
#5182: DeepSeek V4.1 Flash: 1M Context, 437x Smaller KV Cache
DeepSeek V4.1 Flash landed with a 1M-token window and a KV cache 437x smaller than V1. Here's what actually changed — and why the middle of your co...
#5154: DeepSeek's Two-Endpoint Philosophy
DeepSeek quietly routed Pro traffic to Flash — and that routing change says everything about its two-endpoint strategy.
#5044: Abilene's AI Arms Race: Power, Water, and Backlash
Why a Texas town of 100,000 is hosting the biggest AI build ever—and why locals are starting to push back.
#5041: GPT-6 Astra, 100K GPUs, and the AGI Question
OpenAI's GPT-6 Astra arrives with an AGI claim and a 100K-GPU training run. We break down what Stargate LLC actually is and whether the scale justi...
#5004: What Precision Is Your Model Actually Running At?
When you call an API, are you getting the model you think you are? Inference providers quantize more than they admit.
#4973: Building a Pure AI Inference Server
Why your "AI server" should look more like a database than a web app, and how to actually build one.
#4972: CPU vs GPU Inference: When to Skip the GPU
Why running AI on CPU isn't a toy — and when the GPU advantage vanishes.