Local AI
394 episodes · Page 2 of 17
#6444: Teaching Speech Models to Collapse Spelled Words
Fine-tuning formatting rules takes surprisingly little data: a Johns Hopkins study found truecasing accuracy jumps with only fifty documents.
#6443: Why Nvidia Artificially Halves Consumer Tensor Math
A community tool flipped two registers to restore full-rate FP32 accumulation on GeForce cards without overclocking a single clock cycle.
#6433: Why Nvidia Ships Nemotron When Hardware Is the Business
Nvidia spent a trillion distillation tokens on Nemotron Super VL not to top benchmarks, but to hit 4,856 tokens per second on a single GB200.
#6432: Why Speaker Embeddings Fail Across Time and Noise
A standard speaker embedding compresses an entire human voice into just 192 abstract numbers, yet error rates jump from 17% to 47% in the wild.
#6421: Why Diarized Transcription Requires a Cascade
Open-source speech tools cannot transcribe and identify speakers in a single call, forcing pipelines to rely on timestamp-overlap math.
#6409: Why Speech Models Break at Chunk Seams
Shifting an audio transcription window by just one second can lose entire words or drop capitals mid-sentence without changing the model weights.
#6408: Why Speech Model Fine-Tuning Demands Clean Restarts
Fine-tuning only on new vocabulary causes catastrophic forgetting, whereas joint training on old and new data provides the optimal benchmark.
#6407: Designing a Typeface as an Engineered System
Round letters must poke roughly two percent past vertical guidelines to appear the same height as flat letters to the human eye.
#6387: What Really Runs When You Compact an Agent's Context
Commercial compaction relies on main models and prompts, while purpose-built fine-tunes like Compactor-Qwen3.5-4B remain rare.
#6384: Why LLM Output Tokens Cost More Than Input
Decode computes at just 1.4% utilization on an H100 because reading 140 gigabytes of weights takes 280 times longer than the arithmetic.
#6381: Turning Anonymous Diarization Clusters into Names
Shortening an enrolment sample to two seconds can triple the verification error rate compared with full-length recordings.
#6380: Choosing the Right Voice Activity Detector
A compact model file can hide a massive runtime, while Cobra bundles its proprietary detector and engine into just 450 kilobytes.
#6379: Porting Voice Models to Core ML and MLX
Core ML packages separate computation specs from raw weights, while MLX runs on CPU and GPU without touching the Neural Engine.
#6378: Running Speaker Identification Locally on Phones
Exported voice embedding models like CAM++ run offline on mobile devices in packages as small as twenty-eight megabytes.
#6377: How Audio Models Tell Voices Apart
A diarization model can tell two voices apart without understanding a single spoken word or assigning a real name.
#6369: The Most Popular GPUs - Today's Cloud Demand
RunPod charges about $1.34 an hour for serverless RTX 4090 access, while a community pod runs for roughly 34 cents.
#6368: What Does It Take To Train An AI Model From Scratch?
nanochat trains a BPE tokenizer in Rust in roughly 34 seconds, yet it can quietly disadvantage languages like Korean by nearly half.
#6366: Why Training Recipes Matter More Than Model Size
Altering learning-rate timing alone improved global word error rates by twelve absolute percentage points in speech recognition experiments.
#6364: Collecting Speech Data on Outdoor Walks
Walking changes breath timing and phrase length, making outdoor speech recordings crucial for fine-tuning NVIDIA's Parakeet model.
#6355: Why Model Downgrades Are Rarely About Quantization
A caching bug and a shift to medium reasoning effort once reduced Anthropic internal benchmark scores by about three percent.
#6349: Splitting Model Layers Across the Internet
Splitting model layers over a wide-area network yields under two tokens per second until scheduling, speculation, and pipelining step in.
#6348: Splitting AI Workflows Between Local and Cloud
A laptop failed a single video sampling step in 80 minutes, while paid cloud endpoints rendered clips in under 90 seconds for 40 cents.
#6347: Splitting LLM Layers Across AMD and Nvidia GPUs
A two-node llama.cpp setup running a 70B model jumps from three tokens per second on gigabit Ethernet to six on 2.5-gigabit.
#6345: How vLLM Squeezes More Requests into GPU Memory
Early serving engines wasted up to 80% of KV cache memory on unspent reservations; vLLM cut that slack to under 4% using virtual memory paging.