Prompt Engineering
250 episodes · Page 2 of 11
#6131: Judging the Judges: How AI Evaluates Creative Writing
LLM judges match human raters — but the human ceiling is low. What that means for evaluating creative writing.
#6130: Picking a Human Labeling Tool for LLM Benchmarks
Building a human review layer for LLM benchmarks? Here's what Argilla, Label Studio, Prodigy, and Potato actually cost in money and maintenance.
#6111: Translation Models That Know Drama From Contracts
Specialist translation models beat general ones on benchmarks — then lose the human vote. What actually ships when you need a register, not just a ...
#6056: Where Does Speaker Structure Belong in AI Podcast Scripts?
Label-and-parse pipelines break in ways that look like model errors but aren't. Plus what the research actually says about the format tax.
#6048: Under the Hood, October 2026: Seventeen Prompt Versions and Three-Times-Faster Voices
Four weeks of pipeline changes, grounded in the code: grace periods, duplicate detection, Hilbert's rules, echo deletion, and voices nearly three times faster.
#6047: A Human Guest on His Own AI Podcast
Daniel built an AI podcast with cloned voices — then sat down as a guest on it. What six thousand episodes taught him about prompts.
#6043: Write the Spec Before You Pick the Model
Long context, character voice, cheap at scale — three requirements that fight each other. The document that decides the fight.
#6042: Who Grades the Grader? AI Writing Benchmarks Put to the Test
Maths benchmarks can't see whether a model's prose collapses by chapter three. Five specialist leaderboards try — and our own writer is on them.
#6033: Progressive Skills Discovery: When Bigger Context Windows Aren't Enough
Bigger context windows were supposed to make skill discovery moot. Here's why the listing budget still bites at scale.
#6021: Why Chat Templates Break Models Silently
Every chat API flattens your messages into special tokens. Get one delimiter wrong and the model just gets quietly dumber.
#6014: Why Every AI Model Is Secretly a Decoder
Encoder, decoder, or both? The three transformer shapes, why one won, and why BERT still refuses to die.
#6011: Inside LLM Inference: Prefill, Decode, and the KV Cache
One 8K-token sequence on a 13B model eats 6.7GB of KV cache—while the weights are only 26GB. Here's what actually happens per request.
#5989: Twenty Tiny Beats That Make a Podcast Stick
Twenty novel recurring segment pitches — and the test each one has to pass: does it add context, or just noise?
#5949: Inside Deep Research Agents: Claims, Citations, and the 84.7% Orchestrator Problem
Why 84.7% of citation errors originate at the orchestrator — and how a claim ledger fixes it.
#5941: Five Minutes Reading Your Dictation Prompt
A listener's speech-to-text cleanup prompt, read out in full — then stress-tested against what ASR actually gets wrong.
#5922: Teaching a Model Your Voice: SFT vs DPO
Three ways to teach a model your writing voice — and the evidence says one of them quietly makes things worse.
#5858: Routing AI Models: When 3 Rules Beat a Trained Router
A deep dive into AI model routing — and why a learned router may not beat three rules you wrote by hand.
#5857: Building a Scorecard for Our Own AI Writers
Daniel wants a real scorecard for the agent that writes this show — repetition, dialogue quality, and whether the retrieval harness actually works.
#5799: Two Agents, One Approval Gate: Wiring a Dictation Workflow
A dictated prompt becomes an episode — if one agent resolves facts, a second tightens prose, and a human approves the dispatch.
#5777: Lorebooks: The Hobbyist RAG That Beat the Labs
How a chat frontend fork built a full retrieval system — keywords, vectors, recursion — so fictional characters would stop forgetting who they are.
#5775: What "Leaked System Prompts" Actually Mean
Anthropic publishes Claude's system prompt — but not its accessory models. What does "leaking" a prompt actually mean?
#5770: Talking to Your Data: What MCP Doesn't Solve
MCP standardized the pipe. It has no opinion about what flows through it — and that's where the hard part lives.
#5768: Who Actually Draws the Follow-Up Chips?
Those suggestion chips aren't the model talking — they're a second, cheaper inference call plus a rules-based classifier.
#5743: Why Claude Keeps Saying "Prior Art
Daniel noticed "prior art" creeping into his Claude reports. Is it wrong — or just a prestige term the model learned to love?