Training Data
325 episodes · Page 10 of 14
#2076: Is Pure NLP Dead? The Hidden Scaffolding of AI
Modern AI didn't appear from nowhere. Discover how decades of linguistic rules and statistical models built the foundation for today's LLMs.
#2066: The Transformer Trinity: Why Three Architectures Rule AI
Why did decoder-only models like GPT dominate AI, while encoders and encoder-decoders still hold critical niches?
#2064: Why GPT-5 Is Stuck: The Data Wall Explained
The "bigger is better" era of AI is over. Here's why the industry hit a data wall and shifted to a new scaling law.
#2063: That $500M Chatbot Is Just a Base Model
That polite chatbot? It started as a raw, chaotic autocomplete engine costing half a billion dollars to build.
#2060: The Tokenizer's Hidden Tax on Non-English Text
Why does a simple greeting in Mandarin cost more to process than in English? It's the tokenizer's hidden inefficiency.
#2056: Music as Language: The Architecture Behind AI Song Generation
A look at how AI music models use audio tokens, transformers, and diffusion to turn text into songs.
#2016: Andrej Karpathy: The Bob Ross of Deep Learning
Why the most influential AI mind prefers a blank text file to proprietary black boxes.
#2010: Building Better AI Memory Systems
We obsess over AI inputs but treat outputs like Snapchat messages. Here's why that's a massive blind spot.
#1992: The Sovereign Compute Shift: Owning vs. Renting AI Iron
Israel is building a sovereign AI supercomputer with 4,000 Nvidia B200 GPUs to keep startups local.
#1979: When Marketing Swallows the Tech
Is AI the same as Machine Learning? We break down the nested hierarchy of artificial intelligence, from symbolic logic to neural networks.
#1943: The Invisible Math Shrinking AI Models
LZMA, Zstandard, and Brotli are shrinking massive AI models, but how do they actually work?
#1909: The Unbakeable Cake: AI's Copyright Problem
Why can't we just delete stolen data from AI models? It's not a database—it's a baked cake.
#1908: The Web's New Bouncer: When to Block AI Bots
AI bots are crawling the web like a bank heist. Are Cloudflare's new controls protecting your content, or just helping Google?
#1882: The Hidden Human Labor Behind AI
AI isn't free—it costs billions for humans to label data. See why annotation is the real engine behind models like Gemini.
#1855: When AI Makes Game Assets, Who Owns the Art?
From blocky polygons to photorealistic assets, AI is transforming how 3D models are made.
#1839: AI's Data Kitchen: From Hoovering to Fine-Tuning
We go behind the curtain of the AI data pipeline, revealing the messy, multi-billion-dollar war over data curation.
#1817: The Hidden Taxonomy of AI: Why Specialized Models Outperform Giants
Explore the vast ecosystem of niche AI models for computer vision and document understanding, far beyond large language models.
#1810: Why Your TTS Sounds Great in English, Terrible Everywhere Else
English AI voices are polished, but global languages hit a wall. Here's why text-to-speech breaks down for Hebrew, Hindi, and beyond.
#1799: The Original AI Blueprints: BERT & CLIP
Before GPT, two models changed everything. Discover how BERT and CLIP taught machines to read and see the world.
#1739: AI Just Designed a New Life Form
Meet Evo: the 40B parameter AI that writes DNA, designs novel CRISPR systems, and is reshaping synthetic biology.
#1737: Nous Research: The Decentralized AI Lab Beating Giants
Meet Nous Research, the decentralized collective outperforming billion-dollar labs with open-source AI and the self-improving Hermes-Agent framework.
#1712: Five AIs, One Question: A Tiananmen Square Test
We asked five AI models the same question about Tiananmen Square. Their answers reveal a stark divide between Chinese and Western AI.
#1700: Can LLMs Learn Continuously Without Forgetting?
We explore a new approach: micro-training updates every few days to keep AI knowledge fresh without constant web searches.
#1682: Israel's China Dilemma: Cheap Chips, Costly Partners
A geopolitical paradox: Israel leans on Chinese supply chains to fix its cost of living, while Beijing backs Iran and its proxies.