Transformer Architecture
226 episodes · Page 5 of 10
#2913: Hitching a Ride on a Cargo Ship: The Truth
The internet myth vs. the reality of booking passage on a commercial cargo vessel — it's slow, expensive, and vanishing.
#2717: Lower Greenville: From Streetcar Suburb to Food Mecca
How one Dallas street went from farmland to counterculture hub to dining destination.
#2682: Live Retrieval vs. RAG: What an Agent Actually Does
Does every AI conversation create a tiny vector store? We unpack the real tradeoffs between live document fetching and pre-indexed RAG.
#2672: When a Startup Claims to Break the Quadratic Wall
A startup claims linear attention scaling at 12M tokens, beating GPT-5.5 on retrieval benchmarks.
#2622: How Transformers Actually Work: Attention, Tokens, and Context
How one architectural change unlocked chatbots, image generation, and protein folding — explained without the jargon.
#2534: Can AI Generate Diagrams Without Typo Disasters?
Why AI diagram tools still mangle text labels — and what to do about it today.
#2517: How Unsloth Makes LLM Fine-Tuning 2x Faster
Unsloth cuts memory usage by 50-70% and speeds up training 2.2x for models like Llama 3 and Mistral.
#2487: Why AI Calls Everything a "Prediction" (Even Images)
Machine learning calls everything a "prediction" — even generated images. Here's why the terminology matters more than you think.
#2431: The 3 Markets in an AI Trench Coat
GPUs, LPUs, and ASICs: why the best hardware for AI depends entirely on what you're trying to do.
#2426: Why DeepSeek V4's Prose Feels More Vivid Than Claude or GPT
A million-token context window at 2% the KV-cache cost — and prose that actually breathes. Here's what makes V4 different.
#2408: How Backpropagation Actually Unlocks Neural Networks
How error signals flow backward through networks to make learning possible — and why "it's just calculus" misses the point.
#2406: Why Million-Token Context Windows Can't Handle 3 Reasoning Steps
Needle-in-a-haystack is dead. Here's what actually measures whether models can think across long documents.
#2374: How Granular Can MoE Experts Get?
Exploring the limits of expert granularity in Mixture of Experts models—how narrow can segmentation go before efficiency or accuracy suffers?
#2366: Why LLMs Forget the Middle of Long Conversations
Why do large language models struggle with the middle of long conversations? Explore the science behind attention dilution and practical fixes.
#2357: Microsoft's Phi: When Data Quality Beats Model Size
Explore Microsoft AI's Phi family of small language models, designed for edge deployment and high efficiency.
#2355: Why Open-Weight Models Are Winning
Discover how Cogito v2.1 leverages process supervision and MoE architecture to redefine reasoning efficiency in open-weight AI models.
#2350: NVIDIA's Strategic Pivot: From Chipmaker to Model Builder
Dive into NVIDIA’s Nemotron 3 Super, a hybrid MoE model combining Mamba, Transformers, and multi-token prediction for cutting-edge efficiency.
#2349: The 30-Person Lab Outpacing AI Giants
Discover how Arcee AI’s Trinity Large Thinking delivers cutting-edge reasoning at a fraction of the cost, all from a team of just 30.
#2348: Diffusion Models Take on Text Generation
Explore Inception Labs’ Mercury 2, a groundbreaking diffusion-based language model that rethinks text generation and reasoning.
#2314: One Model or Three? Inside Claude's Architecture
What makes Claude’s Haiku, Sonnet, and Opus different? Discover how architecture shapes their unique strengths and weaknesses.
#2312: When Bigger Context Windows Aren't Better
Exploring the real-world impact of massive context windows in AI models, from academic research to codebase analysis.
#2261: The Gap Between AI Output and Art
We assess if AI can truly invent a Tolkien-level language, write a coherent novel, or author an original screenplay—and where the real gaps in crea...
#2188: Is Emergence Real or Just Bad Metrics?
The debate over whether AI models exhibit genuine emergent abilities or just appear to because of how we measure them—and why it matters for safety...
#2164: Why Bigger Context Windows Don't Fix Attention
Frontier models have million-token context windows, but attention degrades well before you hit the limit. New research reveals why bigger isn't bet...