Prompt Engineering
250 episodes
#6499: Why Simple Rules Beat Value Models in Reasoning RL
DeepSeek R1-Zero boosted AIME accuracy from 15.6% to 77.9% using only two deterministic checks and no learned critic.
#6498: Why You Must Read AI Traces Before Writing Evals
Reading just thirty real interaction traces by hand reveals failure modes that generic automated evaluation metrics consistently obscure.
#6491: Agent Skills Break When Secrets Leave Your Laptop
Cloud environments deliberately strip uncommitted dot-env files during session uploads, leaving agent skills without working credentials.
#6458: Why Synthetic Fine-Tuning Data Needs Tooling
Rewriting high-quality data synthetically gives a 3.4-point gain, but running the same process on low-quality data yields just 0.5.
#6407: Designing a Typeface as an Engineered System
Round letters must poke roughly two percent past vertical guidelines to appear the same height as flat letters to the human eye.
#6397: Why Model Benchmarks Fail at Wit
When prompted for a joke over 1,000 times, ChatGPT repeated the same 25 jokes more than 90% of the time.
#6393: Why Model Verbal Tics End Up Driving Script Benchmarks
A pattern search across forty recent episodes found host emotional forecasting in 55% of scripts generated by GPT-6.1 Sol.
#6392: Verifying Generated Arabic Text as a Non-Reader
A study of DALL-E 3 across 2,808 images found its correctness on Arabic letter forms was effectively zero.
#6382: Choosing AI Models on Cost and Performance
Model prices vary thirty-eight-fold per index point, meaning ratio formulas love cheap options regardless of actual quality.
#6310: Why Model APIs Lack Dedicated Few-Shot Fields
Zou and colleagues found models suffered steep drops at 16,000 tokens on tasks requiring learning across an entire demonstration set.
#6294: Why Tool Calls Break Across Coding Models
Translating coding agents across providers fails when input schemas, user-role results, and tokenizer markers diverge.
#6284: Why Coding Agent Configurations Do Not Travel Well
Codex caps combined instruction files at 32 kibibytes, which can crowd out nested rules before an agent even starts work.
#6282: How Steering Messages Reach Active Coding Agents
Steering messages in the Responses API are acknowledged and queued without interrupting running tool calls or rewriting past execution.
#6278: Why Model Benchmarks Miss Small Workflow Friction
Switching Claude Code to Sonnet broke an everyday shell alias request that Opus solved simply by recognizing where to look in local context.
#6242: Training a Model on Your Own Taste
What does it actually take to train a small model on your editorial taste using pairwise preferences and DPO?
#6241: Why Your Synthetic Data Isn't as Diverse as You Think
Fifty-nine percent of one GPT-4 dataset started with "Once upon a time." Correct examples can still make a terrible training set.
#6229: Voice Design: Casting Ogres and Radio Voices
Text-described voices vs. reference audio, and which TTS knobs change who a voice is versus how it sounds.
#6218: Training a LoRA to Write Like You
A Humanizer model stripped every trace of the author. The fix isn't a better Humanizer — it's a LoRA trained on your own writing.
#6153: Decision Models: Typed Answers in One Forward Pass
Clef, Jev, and OpenAI's Decisions API return typed answers with probabilities in a single forward pass — no generated text to parse.
#6139: Which AI Writes Our Scripts Best? We Tested Five
Five models, three scripts each, one rubric. Sol won on length, Grok on voice — and the runner-up title hides a catch.
#6136: How We Chose Our New Script Writer
Ten models audition to write the show. Seven fail the censorship gate — including the workhorse that wrote three episodes in four.
#6135: How Many Runs Does Your AI Eval Actually Need?
Reruns shrink one kind of noise and leave another untouched — so where should your eval budget really go?
#6133: One Rater Is Enough: Designing Daniel's Taste Test
When the ground truth is one person's taste, agreement with strangers is the wrong bar. Stability with yourself is the right one.
#6132: How to Run a Model Eval on a Budget
Daniel is teaching himself to run evals on a real budget. Successive halving, paired comparisons, and why three clean scripts prove almost nothing.