AI Evaluation
522 episodes · Page 6 of 22
#5467: Small Models, Big Schemas: When JSON Constraints Backfire
Small models plus strict JSON schemas should be a safe bet. A 15,000-generation study found the opposite.
#5460: Four Small Models, One Android Phone: Does It Actually Work?
A chained on-device dictation pipeline — VAD, ASR, cleanup — and why "it feels smooth" isn't the same as knowing it works.
#5447: The Models That Never Talk Back
Some models read text, score it, and return a number. No chat, no reasoning, just decisions — and they're running under every router you use.
#5442: When Do LLMs Suddenly Get Good at Things?
New capabilities seem to appear out of nowhere as models scale. Whether that's real — or just how we measure — is still an open fight.
#5438: Why Linguists Left the AI Room
Large language models grew out of linguistics — so why aren't linguists in the room where they're built?
#5437: Hunting the Tells That Vanish When Named
There's no tool for finding an LLM's verbal tics — you have to build one. Here's how keyness analysis works.
#5436: Small Models as Rewriters, Not Writers
Why "don't say X" prompts backfire, and how a tiny grammar-constrained model can scrub a script without breaking its grammar.
#5431: The Other Half of Hugging Face: Why BERT Still Out-Downloads Llama
Encoder models pull over a billion downloads a month. Decoder models pull 397 million. The AI conversation and the download counter are describing ...
#5426: AI Wrote His Yom Kippur Prayer
A man skipped synagogue, opened ChatGPT instead, and got a prayer that knew his life. Is AI-generated prayer real prayer?
#5412: Editing vs. Note-Taking for AI Fine-Tunes
Hand-editing a model's output gives three training signals at once. Writing notes gives one — and a weaker one at that.
#5409: What Makes a Model "Agentic"? Atria Dawn Preview
A 744B-parameter model drops quietly on Hugging Face. Is "agentic" a real model category, or just a deployment pattern?
#5408: When Small NLP Models Beat the LLM
Feature extraction, fill-mask, token classification — the classic NLP tasks still have a job. Here's when a small model beats a frontier API.
#5407: Hemmingway-1 and the War on Waffle
A 27B model promises answers without the preamble. Its benchmark is homegrown — and the behavior it targets has a paper trail.
#5401: When Companies Hide Humans Behind the AI Curtain
A system prompt and a cheap model can make human decisions read like bot output — and that's exactly the point.
#5399: Who's Actually Behind the AI? Fauxtomation Explained
Amazon's Just Walk Out needed 1,000 workers in India. Google Duplex, Facebook M — the pattern of selling humans as software goes deep.
#5391: The Software Behind Urgent Care Triage
Big buttons, emoji vitals, and a system that says "order IV" — what's actually running behind the triage screen?
#5312: Fighting Fake-Personalized Spam With AI Sinkholes
Scraped flattery, fake familiarity, and the one-sided ask — plus the tarpit history behind Daniel's sinkhole agent idea.
#5276: Who Really Invented the Gantt Chart?
Karol Adamiecki built it in 1896 — in Polish. Henry Gantt got the name. And why critical path analysis still catches what humans miss.
#5256: Ruby Won the AI Benchmark. Is It Dying Anyway?
97% of public Ruby repos sat untouched for six months — yet Ruby beat Python in an AI coding benchmark. What's actually declining?
#5255: When AI Writes the Code, Who Checks It?
If agents write the code, the human job becomes knowing when it's wrong. What that means for languages, jobs, and CS degrees.
#5251: Recovering the Moon That's Already in the Pixels
Image restoration clarifies photos without inventing detail — until the fidelity dial shows up. Here's what's in the toolbox.
#5230: Why Your Research Agent Won't Stop Searching
Most agent search runs add no new evidence — and 66-85% of the context window is just search snippets. Here's what actually stops the loop.
#5187: Translation Is Not a Pipe: What Machines Miss
Rubrics rate machine translation above human work — until readers pick side-by-side. What translators know that metrics can't see.
#5183: Where AI Actually Sits on the Hype Cycle
AI has been doing real work for decades. So why do vendors still shout it from the rooftops?