Training Data
325 episodes
#6524: Why Israel Never Finished Its Sovereign Hebrew AI
Israel signed a 37-million-shekel contract for a Hebrew language model 13 months after ChatGPT had already rendered its technical rationale obsolete.
#6523: Unpacking Israel's National AI Strategy
Israel set a national target of 100,000 AI accelerators, but primary documents commit no state budget to building them.
#6508: Why Speech Transducers Omit Common Symbols
A transducer-style speech model cannot learn to emit an at-sign through fine-tuning if the character is missing from its fixed output vocabulary.
#6503: Why Tokenizer Vocabulary Size Is a Hardware Choice
Inference-optimal vocabulary size shifts sixteen-fold depending on serving batch size, from 32,000 at batch one to 524,000 at batch sixty-four.
#6502: What a Cross-Entropy Loss of 1.8 Actually Means
A cross-entropy loss of 1.8 nats means a language model is about as uncertain as choosing uniformly among six equally likely tokens.
#6498: Why You Must Read AI Traces Before Writing Evals
Reading just thirty real interaction traces by hand reveals failure modes that generic automated evaluation metrics consistently obscure.
#6495: Teaching Speech Models to Choose Their Own Audio
Selecting the most uncertain speech recordings often just surfaces silence, clipping, and abandoned half-sentences.
#6494: Where Speech Models Actually Learn
Just three self-attention heads out of twenty in Whisper's decoder trigger over 75 percent of hallucinations on non-speech audio.
#6493: Why Speech Models Forget Other Voices When Tuned on Yours
Adapting Whisper to one person's voice can trigger catastrophic forgetting, stripping away timestamps, segmentation, and accuracy on other speakers.
#6466: Conferences Rule Machine Learning, Not Journals
At NeurIPS 2025, over 21,000 papers required nearly 22,000 reviewers, yet fewer than half a percent earned oral presentations.
#6464: Teaching Speech Recognizers to Stay Quiet
Masking just three self-attention heads in Whisper large-v3 cuts hallucinations on non-speech audio by over eighty percent.
#6463: Why Training Speech Recognition From Scratch Fails
Thirty-two hours of speech provides just three hundred thousand words of text, making from-scratch speech models struggle with basic spelling.
#6458: Why Synthetic Fine-Tuning Data Needs Tooling
Rewriting high-quality data synthetically gives a 3.4-point gain, but running the same process on low-quality data yields just 0.5.
#6457: Where to Learn LLM Training Without Generic Courses
A GPT-2 scale model that took OpenAI 168 hours to train in 2019 can now be replicated on a leaderboard in under two hours.
#6456: The Unsung Builders of Modern Speech AI
OpenAI's Whisper lists famous co-founders among its six authors, yet shipped with no launch event, demo day, or canonical interview.
#6453: Who Actually Won the Open Weight Race
With over four billion downloads, the most downloaded open-weight model in history is an embedding tool, not a chatbot.
#6444: Teaching Speech Models to Collapse Spelled Words
Fine-tuning formatting rules takes surprisingly little data: a Johns Hopkins study found truecasing accuracy jumps with only fifty documents.
#6398: Why Named Entity Linking Fails After the Span Is Found
Zero-shot large language models still lag behind supervised systems on named entity recognition benchmarks by over eleven F1 points.
#6388: Why Swipe Decks Make Better Training Data
A Tinder-style swipe deck removes comparison to force isolated choices, turning rapid binary labels into active learning training signals.
#6375: Why Modern AI Rests on Statistics
When polygenic risk scores derived from European data are applied globally, prediction accuracy drops to 25 percent in African populations.
#6368: What Does It Take To Train An AI Model From Scratch?
nanochat trains a BPE tokenizer in Rust in roughly 34 seconds, yet it can quietly disadvantage languages like Korean by nearly half.
#6366: Why Training Recipes Matter More Than Model Size
Altering learning-rate timing alone improved global word error rates by twelve absolute percentage points in speech recognition experiments.
#6365: Teaching Speech Models to Transcribe Bilingual Slang
Fine-tuning Parakeet with 77 minutes of speech cut Hebrew-English code-switching errors nearly in half, but later gains required far more audio.
#6362: Where Mathematicians Actually Work and How AI Fits In
A surge of over 700 automated AI proof manuscripts hit the mathematics community in October, testing how researchers verify claims.