AI Inference
554 episodes · Page 9 of 24
#4970: Labels vs. Book Covers: Structured Extraction Tradeoffs
General vision models vs. purpose-built pipelines for extracting data from messy real-world labels and covers.
#4967: AI Gateways: Worth the Middleware Tax?
When does an AI gateway's overhead justify its features? We break down the tradeoffs.
#4966: Local vs Cloud: Running Hugging Face Models
Hugging Face's compatibility tracker, cache management, and the real difference between Inference Endpoints and the direct API.
#4958: Multi-Model Transcription with LLM Reconciliation
Using multiple transcription engines and an LLM judge to catch errors in voice-to-text — especially tricky technical terms.
#4920: Local Dictation on Android: The Real Bottlenecks
Why on-device speech-to-text on Android hits a wall at 30 seconds — and what silicon actually matters.
#4919: Parakeet vs Whisper: On-Device Dictation Showdown
NVIDIA Parakeet beats Whisper on phone dictation despite having more parameters. Here's why architecture matters more than model size.
#4836: When AI Actually Touches Money
How Amazon uses LLMs to classify damage and set refunds — and the guardrail stack that keeps the AI from touching the dollar amount directly.
#4834: Why Your AI Bill Keeps Surprising You
How a 37-cent project and a $100 simulation reveal the hidden math of AI API pricing.
#4822: Cloud-Local AI Hybrid: Does It Actually Work?
Can you pair Claude's reasoning with a local Qwen 7B for savings? The gap is wider than you think.
#4792: Shadowing an IMS Forecaster in 2026
What does a human forecaster actually do when AI models already handle 95% of the routine work?
#4789: How LLMs Actually Know When to Stop
Three layers of stop signals, from EOS tokens to context limits — and why none of them really "know" when to stop.
#4788: How Weather Prediction Became a Science
From hand calculations to chaos theory to machine learning — the full arc of forecasting.
#4778: SSH MCP and Tailscale: Docker Migration Guide
How to configure Tailscale for Docker containers when migrating from public IPs to tailnet addresses.
#4774: When a Token Isn't a Word Anymore
Images, audio, and code all use "tokens" — but they're completely different things. Here's how tokenization actually works across modalities.
#4760: Context Window Management: Cutoff vs. Compaction
Trailing cutoff or auto-compaction? How to manage Claude Code's context window without losing critical information.
#4723: Are Model Families Actually Different Models?
Claude Opus, Sonnet, and Haiku aren't trim levels — they're different models sharing a brand name.
#4717: Building a Home Whisper Server: The Full Spec
Daniel wants a dedicated home server for Whisper dictation. Here's the exact build — GPU, RAM, storage, and runtime — to make it work.
#4696: Why AI Over-Explains Simple Tasks
Why does AI turn a simple request into a 24-page document? We explore the training biases and architectural limits behind AI's tendency to over-del...
#4690: Typst for Agents: Programmatic PDF Generation
How Typst's function-based design makes it the ideal document engine for AI agents generating professional PDFs.
#4686: How Claude Code's Patience Works: Fork, Wait, and the Transcript Stack
Claude Code's "waiting" is an illusion. We break down the Unix primitives behind its background task system and where the analogy cracks.
#4683: Parallel vs Sequential Agent Loops: The Real Tradeoffs
When should AI agents run in parallel vs sequential chains? The dependency graph isn't enough — join costs and error handling matter more.
#4669: What "Distribution" Really Means in AI Models
Unpacking what "distribution" actually means under the hood — and why training data variety shapes model behavior.
#4605: Why Chatterbox Still Leads Open-Source TTS
We break down the architecture of Chatterbox, the open-source TTS model powering this podcast, and why it remains state of the art.
#4604: When Sub-Agents Overwhelm the Orchestrator
Daniel's inventory system hit a wall when parallel sub-agents flooded the orchestrator's context window. We break down the math and the fix.