AI Inference
554 episodes · Page 7 of 24
#5853: Who Actually Competes With Hugging Face?
2.96 million model repos, and 85.6% have under 200 downloads. What ModelScope, Kaggle, and Ollama actually do differently.
#5830: Serving Your Own Fine-Tuned Model in the Cloud
You fine-tuned an open-weight model. Now how does anyone actually talk to it? Dedicated GPUs vs serverless inference, and the math that decides it.
#5813: How Python Became AI's Default Language
Python was never designed for science. So how did a general-purpose scripting language end up carrying the entire AI industry?
#5804: Text Classifiers: Local Models vs Always-On Endpoints
A specialist classifier costs $5.70 per million messages. A general LLM costs $142. So why is the obvious product so hard to find?
#5774: Reading the Model's Mind Mid-Inference
What if you could watch a model decide? Inside the tools that open up inference mid-computation — and the limits they hit.
#5773: Declaring Environments: Compose, Nix, and the Agent Boundary
Docker Compose, Nix, devcontainers, microVMs — how automated provisioning works, and where the boundary moves when an agent needs a workspace.
#5768: Who Actually Draws the Follow-Up Chips?
Those suggestion chips aren't the model talking — they're a second, cheaper inference call plus a rules-based classifier.
#5697: Decision Models That Return Probabilities, Not Text
A new class of model skips text generation entirely and returns calibrated probabilities in one forward pass. Here's what that changes.
#5673: Why Lowering Temperature Broke Daniel's Scripts
Lowering temperature should make output safer, not incoherent. Daniel's DeepSeek scripts say otherwise — and the reason may be stranger than he thi...
#5624: One Forward Pass: Building a Bounded Classifier
No generation, no parsing — just label scores in a single pass. How to build a small encoder classifier that actually works.
#5621: Constrained Decoding for AI-Generated Podcast Tags
A working AI podcast pipeline has one quiet failure: the tagging. Here's how constrained decoding and a stable taxonomy fix it.
#5573: Audio Post-Processing for AI Podcast Pipelines
Silence trimming, loudness normalization, EQ, and tempo fixes — the CLI tools that turn raw TTS output into a finished episode.
#5524: Why AI Voice Agents Can't Hang Up
Two polite AI agents, one endless goodbye loop — and a phone bill that never stops climbing.
#5507: Inside ChatGPT's Hidden Pipeline: 8 Models Per Reply
One chat turn isn't one model call. It's moderation, PII filtering, a tone-based router, and activation classifiers — here's the documented graph.
#5506: AI Is Way More Than Chatbots
The word "AI" quietly became a synonym for "chatbot." Here's what the full model ecosystem actually contains.
#5503: Inside the Hidden Image Generation Pipeline
That one-click image generator is secretly a graph of many models. We reconstruct the hidden pipeline behind Gemini and ChatGPT.
#5480: Writing a Personal Instruction for ChatGPT
Daniel's personal ChatGPT prompt gets a line-by-line critique — and the case for shorter, sharper custom instructions.
#5464: Your Keyboard's Hidden Data Problem
Your keyboard knows your email address, your phrases, your habits — and you can't take any of it with you.
#5460: Four Small Models, One Android Phone: Does It Actually Work?
A chained on-device dictation pipeline — VAD, ASR, cleanup — and why "it feels smooth" isn't the same as knowing it works.
#5459: Rebuilding the Podcast: Turn-Taking, Buttons, and Chatterbox
Daniel wants a push-to-interrupt button and a live voice loop. The research says the button is easy and the live part is a research project.
#5457: Pigeon Alerts on a Repurposed i5: Frigate Without the TPU
Daniel wants a pigeon notification, not a custom model. Frigate already ships with bird detection — the real work is masks, zones, and filters.
#5456: Parakeet vs Whisper: Picking a Phone ASR Model
Why Whisper loses on Android, why Parakeet v2 beat v3, and how to benchmark speech-to-text without any tooling.
#5447: The Models That Never Talk Back
Some models read text, score it, and return a number. No chat, no reasoning, just decisions — and they're running under every router you use.
#5435: When Your TTS Model Eats the Numbers
Numbers, dates, and acronyms break text-to-speech in specific, documented ways. Here's where normalization lives — and why it depends on your model.