Local AI
394 episodes · Page 8 of 17
#5436: Small Models as Rewriters, Not Writers
Why "don't say X" prompts backfire, and how a tiny grammar-constrained model can scrub a script without breaking its grammar.
#5434: When Local AI Tools Phone Home
Local AI tools shouldn't need the network. So why is your on-device writing assistant uploading 40MB at 3am?
#5431: The Other Half of Hugging Face: Why BERT Still Out-Downloads Llama
Encoder models pull over a billion downloads a month. Decoder models pull 397 million. The AI conversation and the download counter are describing ...
#5419: The Attention Budget in Your Pocket
Why phone dictation runs out of room, and how bounded attention windows buy you punctuation without blowing your memory budget.
#5411: Fine-Tuning at 4-Bit vs 16-Bit: What It Really Costs
QLoRA cuts fine-tuning VRAM 15x and cost up to 85% — but you pay in training time, quality, and safety alignment.
#5410: Adapters: 102KB That Reshapes a 403GB Model
A 102KB adapter file changes how a 403GB base model behaves — without ever merging into it. Here's how model adapters actually work.
#5397: Chaining Small Models for Voice Cleanup
Six cleanup stages at 97% accuracy each compound to 83% end-to-end. So how many small models can you actually chain?
#5393: Fine-Tuning a Model on 100 Hand-Edited Answers
You don't need 10,000 examples to make a model sound like you. The real number is closer to 100 — if the edits are opinionated.
#5392: Building Agents You Can Actually Move
Agent portability isn't a copy job — it's a rebuild. Why memory, not code, is where lock-in lives.
#5390: Chaining Small Models for Dictation Cleanup
Daniel's Android dictation fork won't render "three point five" as a decimal. How many models does cleanup actually need?
#5389: Hugging Face vs Kaggle: Where Models Actually Live
Hugging Face and Kaggle aren't rivals — one is infrastructure, one is a practice field. Here's how the two platforms actually differ.
#5388: Android's Split Keyboard and Voice Input Problem
Android separates keyboards from speech engines by design. Most apps weld them back together.
#5387: Who Decides What Your Keyboard Knows?
Your keyboard's word list was probably frozen in 2014. Here's who maintains the open-source dictionaries — and why Gboard's stays fresh.
#5384: Android ASR Runtimes: LiteRT, ExecuTorch, and Why Your Phone Has No VRAM
Why does your phone have no VRAM number? A tour of Android's runtime layer and what it takes to run ASR locally.
#5245: Training a Model on 100 Phone Photos
One drawer of screws, a hundred phone photos, and a schema. When does a small custom model beat a fine-tuned VLM?
#5188: DeepSeek's Point Release That Isn't
DeepSeek shipped a whole new architecture and called it a point release. Here's what actually changed inside the model.
#5154: DeepSeek's Two-Endpoint Philosophy
DeepSeek quietly routed Pro traffic to Flash — and that routing change says everything about its two-endpoint strategy.
#5054: Hardware Clubs for Grownups: Finding Your People
Software is solitary. Hardware is social. Here's how to find or build a space where adults and teens build together.
#5012: Masked Safety: How Post-Training Changes AI Behavior
Safety mechanisms aren't erased in post-trained models—they're masked. Here's how that changes everything for military AI.
#5004: What Precision Is Your Model Actually Running At?
When you call an API, are you getting the model you think you are? Inference providers quantize more than they admit.
#5000: Can You Trust Google Photos With Your Life's Work?
Why your Google Photos library isn't really backed up — and what actually works instead.
#4981: The Void That Doesn't Heal
Why the grief of not having supportive parents feels different — and what neuroscience reveals about it.
#4973: Building a Pure AI Inference Server
Why your "AI server" should look more like a database than a web app, and how to actually build one.
#4972: CPU vs GPU Inference: When to Skip the GPU
Why running AI on CPU isn't a toy — and when the GPU advantage vanishes.