AI Evaluation
522 episodes · Page 11 of 22
#2664: Can You Trust an LLM's Raw Knowledge?
Why pre-trained knowledge isn't reliable for facts — and what actually makes models useful.
#2651: AI Training Itself: Student, Teacher, and Grader
Can models generate their own training data and judge their own outputs? The promise and pitfalls of fully AI-led pipelines.
#2650: How to Catch an LLM's Bad Writing Habits
A practical guide to analyzing podcast transcripts for repetitive language and dialogue patterns — from Python word counts to embedding clustering.
#2648: The Art of the Brief: Writing What Busy People Actually Need
Why a crisp 600-word brief is harder than a 10-page report—and how AI changes the game.
#2643: How Stenographers Type 300 Words Per Minute
Court reporters don’t type letters—they chord syllables at 300 words per minute. Here’s how it works and why AI can’t replace them yet.
#2640: Why Instructional Models Beat Conversational for Batch AI
Beyond cheaper tokens—how batch inference changes AI workflows and why instructional models beat conversational ones for automated jobs.
#2639: The Hidden Layer That Makes Search Work
Why your search results miss the mark — and how cross-encoders fix it.
#2612: Can You Speed-Date Your Way to the Right Therapy?
Finding the right therapy is a guessing game. What if you could sample different approaches before committing?
#2609: Mapping the Therapy Family Tree: CBT, ACT, DBT & Beyond
How CBT, ACT, and DBT actually evolved — and why matching therapy to personality matters.
#2586: Pseudo-Personalized Emails: The New Spam Uncanny Valley
How to detect and filter AI-generated outreach emails that fake personal connection without nuking legitimate messages.
#2578: Building Deliberately Slow Deployment Pipelines
How to build CI/CD pipelines designed as filters, not firehoses — with manual gates, staging environments, and quality checks.
#2551: How Progressive Disclosure Saves MCP from Token Bloat
Why dumping all tool schemas into context breaks accuracy — and three implementations that fix it.
#2549: Jakob's Law: Why Users Think Your App Is Broken
Why broken keyboard shortcuts destroy user trust — and what Jakob's Law reveals about design expectations.
#2544: How to Make AI Architectural Renders Photoreal Without Breaking Geometry
Fixing the uncanny valley in AI-enhanced architectural renders — without breaking the geometry.
#2539: When Does AI Stop Hallucinating and Start Reconstructing?
What happens when you feed hundreds of photos into an AI world generator — do you capture reality or just a convincing dream?
#2516: Overfitting Is Not a Binary Condition
Overfitting isn't binary. Learn the real triggers, the bias-variance tradeoff, and modern techniques to prevent it.
#2511: Measuring AI API Latency Through the Black Box
How to benchmark token throughput and debug slowdowns in closed CLI tools like Claude Code using OpenTelemetry and mitmproxy.
#2510: The Design That Makes Voice Agents Tolerable
Drive-thru accuracy, healthcare triage, and the design secret that makes people *want* to talk to a machine.
#2494: Active Prompt Engineering: Daniel's Diff-Based Loop
A deep dive into iterative prompt refinement using inter-iteration prediction change as an uncertainty signal.
#2483: Substitution Anonymization: Privacy Without Utility Loss
How to generate realistic synthetic voice notes and calendar data with zero PII exposure risk.
#2471: Creative Briefs for AI Agents: What Agencies Already Know
How agency best practices for briefing creatives map directly onto getting reliable output from AI agents like Claude Design.
#2468: When Tokens Meet GPU Seconds
How to track AI spend across Open Router, Replicate, and more — without a unified dashboard.
#2443: How Podcast RSS Feeds Can Speak Every Language
One RSS feed, a transcript tag, and TTS voice cloning — the emerging standard for letting any podcast speak any language.
#2419: Methylation vs. IEMs: Untangling the Confusion
Methylation isn't a health dial. Learn how it actually works in the body vs. rare genetic IEMs.