Local AI
394 episodes
#6517: Why GPU Wiring Dictates Model Parallelism
Crossing from NVLink inside a server to InfiniBand between servers drops available bandwidth by a factor of eighteen.
#6514: Why Even Solo Developers Face a GPU Shortage
H100 rental contracts jumped nearly 40 percent in five months as on-demand GPU capacity sold out across legacy and cutting-edge hardware.
#6510: Voice Isolation Before Speech Recognition
Running voice isolation on clean speech without a competing talker increases transcription word error rates from 0.04 percent to nearly three percent.
#6508: Why Speech Transducers Omit Common Symbols
A transducer-style speech model cannot learn to emit an at-sign through fine-tuning if the character is missing from its fixed output vocabulary.
#6503: Why Tokenizer Vocabulary Size Is a Hardware Choice
Inference-optimal vocabulary size shifts sixteen-fold depending on serving batch size, from 32,000 at batch one to 524,000 at batch sixty-four.
#6496: How Speech Models See Audio as an Image
Modern speech recognition models like Whisper never hear raw audio; they inspect an eighty-pixel-tall grayscale image of the sound.
#6494: Where Speech Models Actually Learn
Just three self-attention heads out of twenty in Whisper's decoder trigger over 75 percent of hallucinations on non-speech audio.
#6493: Why Speech Models Forget Other Voices When Tuned on Yours
Adapting Whisper to one person's voice can trigger catastrophic forgetting, stripping away timestamps, segmentation, and accuracy on other speakers.
#6492: What Audio Models Learn Before Fine-Tuning
With just ten minutes of labeled audio on top of unlabeled pretraining, wav2vec 2.0 achieves a 4.8 word error rate on clean speech.
#6486: Why Voice Satellites Need Integrated Audio
Separating a voice satellite's microphone from its speaker eliminates hardware echo cancellation, making seamless follow-up questions impossible.
#6484: Why Narrow Speech Tasks Still Need Large Models
NVIDIA experiments show speech models trained from scratch on target domains had nearly double the error rate of fine-tuned pretrained models.
#6475: USB Audio Limits on Android for Voice Models
Android caps native USB audio capture at 48 kilohertz, leaving higher rates entirely to apps packing their own custom drivers.
#6473: Why Whispering Breaks Speech Recognition Physics
Whispering completely removes fundamental frequency and pitch, forcing speech recognition models to rely on subtle duration cues.
#6469: Why Switching Off One Attention Head Changes Everything
Masking just three decoder heads in Whisper drops hallucination on silent audio clips from 99.97% down to 24.1%.
#6468: Why Tiny Neural Gates Replaced Voice Activity Gates
Legacy WebRTC voice detection misses sixty-four percent of speech frames, while modern neural models compress full memory into a single megabyte.
#6467: Voice Activity Detectors Need Adaptive Thresholds
When speech recognizers receive pure noise from a loose VAD gate, they confabulate plausible sentences rather than returning silence.
#6465: Why Modern AI Models Cut Most Attention Layers
Frontier models like Kimi K3 now replace three out of four attention layers with delta-rule recurrence to curb massive KV caches.
#6464: Teaching Speech Recognizers to Stay Quiet
Masking just three self-attention heads in Whisper large-v3 cuts hallucinations on non-speech audio by over eighty percent.
#6463: Why Training Speech Recognition From Scratch Fails
Thirty-two hours of speech provides just three hundred thousand words of text, making from-scratch speech models struggle with basic spelling.
#6457: Where to Learn LLM Training Without Generic Courses
A GPT-2 scale model that took OpenAI 168 hours to train in 2019 can now be replicated on a leaderboard in under two hours.
#6456: The Unsung Builders of Modern Speech AI
OpenAI's Whisper lists famous co-founders among its six authors, yet shipped with no launch event, demo day, or canonical interview.
#6453: Who Actually Won the Open Weight Race
With over four billion downloads, the most downloaded open-weight model in history is an embedding tool, not a chatbot.
#6451: Which AI Models Actually Last
OpenAI kept the gpt-3.5-turbo alias alive for over three years, while Google Gemini Flash iterations often turn over in just a few months.
#6448: Why Fast Model Microtraining Needs an Evaluation Gate
A speech model fine-tuning run can cost twenty-three cents in twenty minutes, but unmonitored updates silently erase older vocabulary.