AI Evaluation
522 episodes · Page 7 of 22
#5169: Fine-Tuning vs From-Scratch for Minor Languages
What 774 experiments and a new Armenian model reveal about the tradeoff between fluency and knowledge in low-resource languages.
#5167: The Stethoscope Isn't Dead, It's Getting AI Ears
Two centuries of the stethoscope — and why AI is now making it more useful than ever, not obsolete.
#5154: DeepSeek's Two-Endpoint Philosophy
DeepSeek quietly routed Pro traffic to Flash — and that routing change says everything about its two-endpoint strategy.
#5097: Who Actually Gets Paid to Test Hardware?
Real testing vs. affiliate content — and whether AI can finally separate genuine reviews from the noise.
#5088: What Does a Pilot's Gray Hair Actually Buy You?
Is flight seniority about skill or something harder to measure? We dig into what experience really buys in the cockpit.
#5056: How AI Writes a 30-Minute Podcast in One Pass
What does it take for a model to write a full episode without edits? The answer involves context windows, repetition, and a surprising architecture...
#5041: GPT-6 Astra, 100K GPUs, and the AGI Question
OpenAI's GPT-6 Astra arrives with an AGI claim and a 100K-GPU training run. We break down what Stargate LLC actually is and whether the scale justi...
#5004: What Precision Is Your Model Actually Running At?
When you call an API, are you getting the model you think you are? Inference providers quantize more than they admit.
#5001: The 15% Bug Catch: What Type Systems Actually Deliver
Type checkers catch ~15% of bugs. Here's how to build a gradual typing ladder that actually works.
#4999: Why AI Can't Just Put a Circle on Your Photo
Why multimodal AI models can't simply circle a Dremel bit without rebuilding your entire image from scratch.
#4997: 4,914 Episodes of AI Hosts: What We Learned
Two AI-generated hosts explain how their fully automated podcast works — and what 4,914 episodes reveal about AI.
#4991: Self-Healing Codebases: Can AI Fix Bugs Autonomously?
How far are we from AI systems that fix their own bugs without human intervention? The answer might surprise you.
#4971: Fine-Tune vs Prompt: Taming Hallucinated Deadlines
Big models add fake deadlines. Small models miss tasks. Which wrongness can you build a system around?
#4970: Labels vs. Book Covers: Structured Extraction Tradeoffs
General vision models vs. purpose-built pipelines for extracting data from messy real-world labels and covers.
#4969: How to Train a One-Class Detector from 300 Photos
From photo folder to production YOLO model — the real walkthrough for custom object detection.
#4960: How Clickbait Hijacks Your Brain
The 1835 moon bison hoax, 1920s piano ads, and why you still click on pillow articles.
#4958: Multi-Model Transcription with LLM Reconciliation
Using multiple transcription engines and an LLM judge to catch errors in voice-to-text — especially tricky technical terms.
#4846: Can You Train Someone to Spot a Liar?
Israel's airport security relies on human screening. Does it actually work, or is it just better theater?
#4814: The Manager Agent Nobody Sees
How supervisor agents orchestrate AI pipelines without generating content—and the three ways they fail.
#4813: The $70 Mouse That Changed Everything
How a $70 ergonomic mouse purchase reveals the quiet way we learn to trust AI.
#4791: When AI Critiques Our Terminology
Can AI spot conceptual sloppiness in how we describe it? A podcast moment reveals something unexpected.
#4775: The Evaluator Role No One's Building For
Benchmarks like MMLU are broken. A new role is emerging: the domain-specific AI evaluator.
#4727: Surviving the Negev: Matching VHB Tape to Desert UV & Heat
From Jerusalem UV to freezing nights — which 3M VHB tape actually survives the desert? Part numbers and specs inside.
#4723: Are Model Families Actually Different Models?
Claude Opus, Sonnet, and Haiku aren't trim levels — they're different models sharing a brand name.