Daniel's been poking at something that doesn't get enough attention. He's been running this podcast's generation pipeline as what he calls a true agentic system, and the part he keeps coming back to is the research sub-agent. The thing that goes out, crawls the web, and comes back with a dossier the script-writing agent then works from. He's cycled through Gemini's native search, Perplexity, Tavily, and now Exa as the grounding layer. His question is what actually makes a deep research framework worth using over the naive approach, which is just handing an agent a search tool and telling it to go look things up when it needs to.
And the naive approach fails in a very specific way that's easy to miss until you've watched it happen a few times. The agent searches, gets results, searches again, gets slightly different results, and then just... keeps going. It doesn't know when to stop because nothing in its loop is telling it whether the new information is actually new.
Daniel mentioned turn-taking limits as the mitigation. You just cap how many searches the thing can run.
Right, and that's the blunt instrument. It bounds the cost, but it doesn't detect convergence. There's a paper from August that looked at this directly. They instrumented long-horizon search agents and found that between seventy-seven and ninety-four percent of search episodes add no new evidence. The agent is running additional searches and the retrieval set isn't changing in any way that matters.
So most of the searching is theater. The agent looks busy.
It looks busy and it's burning tokens. The same group found that accuracy tracks retrieval recall almost perfectly, a correlation of point nine nine. But search volume is anti-correlated with accuracy at negative point seven seven. More searching doesn't just fail to help, it's actively associated with worse outcomes.
That's the part that should make anyone running a production pipeline sit up. The thing you're paying for per search is making the output worse on average.
And the context window is filling up with search snippets, not with the actual documents. Sixty-six to eighty-five percent of the context in these agents is just the snippets the search API returned. So the quality of your grounding is being determined less by what got fetched and more by how the search API formatted and deduplicated those little preview blobs.
Which is where the provider choice starts to matter in a way that isn't obvious from the marketing. Exa maintains its own index and uses embeddings to match semantic meaning rather than scraping Google. Tavily built its reputation by speaking directly to agent developers and shipping dedicated map and crawl endpoints. Perplexity's deep research is just a model name on a chat completions endpoint.
That last one is the adoption cost play. If you're already calling a chat completions API, switching to Perplexity Sonar Deep Research is a string change. You don't build a job queue, you don't poll a research ID. But you also don't get the intermediate steps. The loop is happening inside their black box.
And that's the trade Daniel's really asking about. What do you get from a framework that hands you back the sources and lets your original agent query again, versus one that just returns a summary?
The serp.fast guide from July put it cleanly. A search API executes the query you gave it and returns results in hundreds of milliseconds. A deep research API takes the query as an objective and decides the searches itself. It proposes sub-questions, searches, fetches pages, notices a gap, searches again, then writes. The unit of work becomes variable, so the price is variable. Latency goes from milliseconds to minutes. And citations become part of the contract.
Citations as contract is the phrase that matters. If you're building anything where a user can ask where did this come from, the attachment of sources to output is most of what you're buying.
Several of these vendors return structured JSON with per-field citations rather than a prose blob. So the generation agent can say this claim maps to this URL, and if it needs to drill into that URL, it can query again. The failure mode of a research endpoint that doesn't give you that is a plausible report with a bad source buried in it, which is considerably harder to catch than an inspectable loop.
Daniel's been listening for exactly that. He said when we do an episode about something in the news yesterday, the information by definition has to come from search or grounding. If the dossier has a bad source buried in it, he hears it in the script.
And the quoting problem is enormous. Tian Pan's writeup from April put the number at fifty to ninety percent of LLM-generated citations not being fully supported by the sources they reference. Even retrieval-augmented systems with web search produce unsupported statements roughly thirty percent of the time.
Thirty percent is a lot of wrong sentences making it into a final script.
And it compounds. An incorrect claim retrieved in iteration two can redirect the entire research trajectory. The agent spends iterations three through eight building on a false premise. That's the real cost of a bad citation. It's not just one wrong sentence, it's the whole downstream investigation getting pointed the wrong direction.
So the naive approach fails in two directions at once. The agent either searches forever because nothing tells it to stop, or it latches onto a bad source early and builds a confident wrong answer on top of it.
The infinite loop problem has a name now. Infinite Agentic Loops, IALs. A paper from HUST scanned six and a half thousand real LLM agent repositories and found sixty-eight confirmed IAL failures across forty-seven projects, at ninety-one point nine percent precision. And the key insight is that frameworks already ship bounds. LangChain has max iterations, LangGraph has recursion limits, OpenAI Agents SDK has max turns, CrewAI has max iter.
So the bounds exist and people still loop. What's going wrong?
The bounds aren't covering the actual feedback path. Developers omit them, misuse them, configure them with ineffective values, or place them outside the loop that's actually running away. The presence of a loop isn't the problem. Whether an effective bound covers that loop's feedback path is the problem.
And then there's fan-out. That's the one that scares me for a pipeline like Daniel's.
Multigrid's writeup on the unbounded agent separates three runaway modes. Step count is the classic one. Spend is the second, because cost per step rises roughly with the square of the run length, so a twenty-step limit doesn't bound spend. And fan-out is the third. If your budget is per-agent rather than per-task, spawning sub-agents multiplies the budget. A fifty-cent cap becomes a fifty-dollar run.
There's a concrete example of this from June. A single deep research call in Claude Code spawned a hundred and eleven agents, burned one point two million tokens, and exhausted a five-hour quota in twenty-six minutes. Zero output.
The postmortem is worth reading. The fetch budget was bypassed by high-relevance URLs, so twenty-nine pages got fetched instead of the fifteen cap. There was no token budget guard. And the verify agent retry loop spawned replacements instead of aborting. Seventy-five verify agents each ran their own independent web search.
The formula was one plus six plus twenty-nine plus twenty-five times three plus one. A hundred and eleven agents from a single call.
And the commenter who diagnosed it said the compounding cost multiplier has nothing to do with the original fetch budget. Each failed verify agent spawns a replacement, each replacement independently searches, and the whole thing spirals.
Daniel's research sub-agent is most at risk from that pattern, not from a single long loop. If it ever starts spawning parallel sub-agents, each with its own turn limit, the budget is now multiplied across the fan-out.
ByteDance hit a related problem in their deer-flow framework. Deep research subtasks were dying with a graph recursion error at a hundred turns. Their fix raised the sub-agent max turns from a hundred to a hundred and fifty and doubled the timeout from fifteen minutes to thirty. And the reviewer noted that at the observed pace, a hundred and fifty turns can exceed the old fifteen-minute timeout, so a turns-only bump would just shift the failure from recursion to timeout.
So the turn limit and the timeout are coupled. You can't raise one without thinking about the other.
And the lead agent's recursion limit of a hundred stayed separate. That's a distinct budget. The structural fix, according to the PR author, is finer task decomposition. Even a hundred and fifty turns may not be enough for briefs that large.
Which brings up the question of what the loop should actually be doing. If more searching doesn't buy accuracy, what does?
The diagnosis paper found that accuracy tracks retrieval recall, not search volume. So the lever is better queries, not more queries. And the failures split into two categories. Retrieval gaps, where the evidence never surfaced at all, and utilization gaps, where the evidence surfaced but the answer was still wrong. The fix for the first is better query formulation. The fix for the second is better verification.
And agents at nearly identical accuracy can sit at opposite ends of that balance. One is failing because it can't find things, the other because it can't use what it found.
The RAAC paper from August tried to address this with what they call retrieval-aware agent control. Four unsupervised signals. Criteria coverage, document novelty, successive query diversity, and original question similarity. The last one is a drift guardrail, making sure the agent hasn't wandered off the original question.
Drift is a real problem. The agent starts researching one thing, finds something adjacent and interesting, and three searches later it's building a dossier on the wrong topic.
And the agent doesn't notice because it lacks awareness of what it's already collected and whether information discovery has saturated. That's the RAAC authors' phrasing. The majority of iterations contribute little or no improvement. They call it reasoning stagnation.
Their controller decides between continue, intervene, or stop. What did that buy them?
On BrowseComp-Plus, they reduced search calls by an average of fourteen, up to forty-four, while gaining up to ten percent accuracy and three percent on average. Fewer searches, better answers. That's the evidence-driven stopping argument.
So the turn limit Daniel's using is the backstop, but the real win is stopping when the evidence says stop.
Tian Pan's line was that the core problem isn't building the loop. It's knowing when the loop should stop. The most robust implementations combine coverage checklists with budget-based cutoffs. Aim for completeness, but guarantee termination.
And Google's Gemini Deep Research runs eighty to a hundred and sixty searches per task. That's the overkill end of the spectrum.
Which connects to a weird tension in the literature. There's a synthesis note from February claiming deep research follows the same scaling curve as reasoning tokens, monotonic improvement then degradation. Search has its own overthinking variant. But the August diagnosis paper found no correlation between search volume and accuracy. Those are in direct tension.
One says more search helps up to a point then hurts. The other says search volume just doesn't matter. Which is it?
I think the resolution is that search volume is a proxy for something else. Bad agents search a lot because they're flailing. Good agents search less because their queries are better and they stop when they've got the evidence. The volume isn't causing the bad accuracy, it's a symptom of the same underlying problem.
That's a useful distinction. The metric you're measuring is downstream of the thing you actually care about.
And the thing you actually care about, in Daniel's pipeline, is whether the dossier that reaches the script-writing agent is grounded in real sources that say what the script claims they say.
Which is the quoting problem again. Fifty to ninety percent unsupported citations.
The defenses are multi-source corroboration, source-type weighting, contradiction detection, dynamic credibility scoring. There was a Hacker News project in February called Librarium that fans queries out to many providers in parallel and outputs a sources file that ranks citations by how many providers independently cited the same URL.
That's a clever signal. If four different search APIs all surface the same source for a claim, it's probably real.
And it's a mitigation for the bad-source-buried-in-a-report problem. The report looks plausible, but if you can see that only one provider surfaced that URL and it's a blog nobody else indexed, you know to weight it lower.
There's a free loss in the diagnosis paper I want to flag. Format and subtle errors accounted for roughly eighteen to twenty percent of wrong answers for the best agents. Markdown wrapping, capitalization. Stripping markdown and normalizing case would recover that fraction without touching search or reasoning.
That's the kind of thing that makes you want to check your own pipeline for dumb losses before you go optimizing the smart parts.
Daniel's prompt mentioned that the research agent returns a summary rather than feeding all the raw sources to the generation agent. That's the architecture. Crawl wide, summarize, pass the summary.
And the summary is where the fidelity gets lost. The research agent reads twenty pages and writes a two-page digest. The generation agent only sees the digest. If the digest drops a nuance or misattributes a quote, the generation agent has no way to know.
Unless the framework returns sources so the original agent can query something again. That's the loop Daniel was describing.
And that's the argument for keeping the loop in your own code. When you need the intermediate steps in your product, showing which sources were considered, not just which were cited, a research endpoint's black box is a liability. The inspectable agentic loop lets you see the search trail.
The counter-argument is that running your own loop means you're responsible for all the bounds, all the convergence logic, all the fan-out control. You're building the thing the vendors already built.
And the vendors have different shapes. Parallel publishes nine task tiers from five dollars per thousand to twenty-four hundred per thousand, latency from ten seconds to two hours. They're the only shape that lets you enforce a budget from the request side. Valyu indexes arXiv and PubMed and SEC filings alongside the open web, async by default with webhooks. Exa's research is a mode on an index rather than a separate product, with an async task API that returns a research ID you poll.
The pricing spread is wild. Five dollars to twenty-four hundred per thousand tasks.
Perplexity's pricing is four token streams. Input, output, citation, reasoning. Plus five dollars per thousand searches. Real calls land between forty-one cents and a dollar thirty-two, with one worked example attributing about a dollar of a dollar thirty-two call to reasoning tokens alone.
The reasoning tokens are the expensive part. The model thinking about what to search next costs more than the actual searching.
Multi-agent research systems use about fifteen times more tokens than a standard chat interaction. Sessions run two to five dollars. That's the hidden cost of the loop. It's not the search API calls, it's the model calls deciding what to search.
That's why the turn limit matters for cost even if it doesn't improve accuracy. Every turn is a model call, and the model call is where the money goes.
The serp.fast guide said the comparison to run is cost per accepted answer, not cost per call. If a cheap search API returns garbage that your generation agent then has to fix, the cheap API was expensive.
Daniel's been through four providers. He's got the practical experience to know which one gives him the dossier quality he needs. The question is what he's actually evaluating when he listens for accuracy.
He's evaluating the whole pipeline end to end. The research agent's query formulation, the search API's snippet quality, the summarization fidelity, the citation attachment, and the generation agent's use of the dossier. Any one of those can fail and he hears it in the episode.
The failure pattern sound different. A retrieval gap sounds like the episode is missing a key fact that any decent search should have found. A utilization gap sounds like the fact is there but the script gets it wrong. A bad citation sounds like a claim that doesn't hold up when you check the source.
The utilization gap is the one that's hardest to diagnose from listening alone. The dossier had the right information, but the generation agent ignored it or misused it. You can't tell that from the outside unless you have access to the intermediate steps.
Which is the argument for the inspectable loop again. If Daniel's research sub-agent logs its search trail and the sources it considered, he can go back after a bad episode and see where it went wrong.
The snippet management point matters here. Since sixty-six to eighty-five percent of context is search snippets, the grounding quality hinges on how the search API returns and deduplicates those snippets. Exa's metadata-rich, model-ready results are designed for exactly that. The snippet is formatted to be useful to a model, not to a human scanning a results page.
That's a subtle point. The search API that's best for a human user isn't necessarily best for an agent. The agent needs structured metadata, clean text, deduplicated results. A human can handle a messy results page. A model gets confused by it.
No public benchmark tests what happens when target sites start blocking AI crawlers. That's directly relevant to Daniel's question about finding available sources from the internet. If the sites you need are blocking the crawler your provider uses, the research quality degrades and you might not even know why.
There was a Hacker News thread two days ago about Google's anti-scraping update. DuckDuckGo blocks most queries now, pushing developers toward Tavily and Exa API endpoints. The landscape is shifting under the providers.
The benchmark situation is a mess. The public evidence about which provider is best is currently two vendor-published benchmarks pointing in opposite directions. DRACO was published by Perplexity researchers from de-identified Perplexity traffic. The most-cited scoreboard was published by Valyu, which ranks first. Parallel publishes its own BrowseComp claims against Exa and Perplexity.
Everyone's benchmark says they win. Shocking.
Treat all rankings as directional. And DRACO doesn't measure schema-fill accuracy, latency, cost per task, tail behavior, or refusal rates. It says little about the job most pipelines actually need, which is filling a JSON schema with grounded data.
Daniel's pipeline needs the dossier to be accurate, not the report to be pretty. The benchmark that matters is whether the episode passes his accuracy check.
The episode is the ultimate end-to-end test. If the research agent retrieved from a variety of sources and the script reflects that, the pipeline worked. If the script has a confident wrong claim, something in the chain failed.
The naive approach fails because it gives the agent a tool and a vague instruction. Make sure you search for things when relevant. No convergence criteria, no budget object, no fan-out control, no citation contract.
The framework's added value is all the things that make the loop terminate at the right time, on the right answer, for the right cost. That's Tian Pan's phrasing. The loop is easy. Making the loop terminate correctly is the real engineering.
The budget object point from Multigrid. A budget should be a property of the task, not the agent. If a sub-agent is spawned, it receives the same budget object, not a fresh one. That's how you prevent the fan-out multiplication.
The model deciding is a heuristic. A bound is a guarantee. You want both, and only one of them can be relied on when the heuristic is the thing that has gone wrong.
That's the line that sticks with me. When the agent's judgment fails, the bound is what saves you. If you don't have the bound, you have the Claude Code incident.
A hundred and eleven agents, zero output, five-hour quota gone in twenty-six minutes.
Hilbert, you've been running search infrastructure longer than either of us. What's the actual difference between a search API and a deep research framework when you're the one paying the bill?
Hilbert: I own one. Several, actually. The difference is the search API does what you tell it. The research framework does what it thinks you meant. That second one costs more and you find out whether it was right when the invoice arrives.
The invoice arrives either way.
Hilbert: The invoice arrives faster with the framework. I ran a crawl operation for a price comparison site back around two thousand eighteen. We had a search budget per item. Three queries, then you take what you got and move on. The framework wanted to do twelve queries per item because it kept finding new sub-questions. We turned that off after the first month.
What did it actually get you, the extra searching?
Hilbert: Nothing. The first three queries had the answer. The other nine were the model convincing itself it needed to look harder. Same products, same prices, same sources. It just felt incomplete to the model.
That's the stagnation finding. The majority of iterations add no new evidence.
Hilbert: I didn't need a paper to tell me that. I had a bill that said it.
The turn limit as backstop, then. You were already doing the right thing.
Hilbert: We had a hard stop because the client had a hard budget. You learn to make the loop terminate when the money runs out. The framework's job is to make it terminate before the money runs out, on the answer you actually needed.
The citation attachment. Did you have to deal with sources being wrong?
Hilbert: We had a sources file that mapped every price to every URL. If a price was wrong, we could trace it. That's the contract. Without that file, you're just trusting a summary and hoping.
The summary is where the fidelity dies.
Hilbert: The summary is a lossy compression. You can't audit a lossy compression. You can audit a sources file. We kept the file for seven years. Nobody ever looked at it except when something broke. But when something broke, it was the only thing that let us fix it.
The inspectable loop argument. You don't need the intermediate steps until you do, and then you really need them.
Hilbert: The framework that gives you the sources is the one you can fix. The one that gives you a paragraph and a confidence score, you're just hoping.
What would you tell Daniel to watch for when he's listening to the episodes?
Hilbert: Watch for the episode that sounds right but isn't. The wrong facts are easy to catch. The plausible wrong facts are the expensive ones. If the research agent pulled a bad source and the script built on it, you won't know until someone checks the source. So check the source.
The fifty to ninety percent unsupported citation number.
Hilbert: I believe it. We had a crawler that pulled product descriptions. Half of them were wrong in ways you couldn't tell without opening the actual page. The snippet looked fine. The page said something else.
The snippet problem. Sixty-six to eighty-five percent of context is snippets.
Hilbert: The snippet is a promise. The page is the delivery. Sometimes the promise is a lie.
The generation agent only sees the promise.
Hilbert: Then the generation agent writes a confident lie and nobody knows until the listener checks.
The one thing I'm taking from this is that the bound is the guarantee and the heuristic is the gamble. Daniel's turn-taking limits are the right instinct, but the real win is evidence-driven stopping. When the research agent can tell that the new searches aren't adding anything, that's when the framework earns its keep.
The thing that sharpens it for me is the anti-correlation. More searching doesn't just fail to help, it's associated with worse accuracy. The agents that search less and query better are the ones getting it right. That's the counterintuitive finding that should change how anyone builds these pipelines.
The loop is easy. Making the loop stop at the right time, on the right answer, for the right cost, that's the engineering.
The one open question I keep turning over is whether the search budget law tension ever resolves. One line of work says search follows the same overthinking curve as reasoning tokens. The other says volume just doesn't correlate with accuracy. I think the synthesis is that volume is a symptom, not a cause. Bad agents search a lot because they're lost.
That feels right. The search isn't the disease, it's the fever.
Thanks to Hilbert Flumingtop for producing. This has been My Weird Prompts. If you want to reach us, email us at show at my weird prompts dot com. We'll be back soon.
See you tomorrow.
