Daniel wrote to us with something that's been rattling around my head since I read it. He says — and I'm going to read this nearly in full because the framing matters — in our episode about the strange paucity of tools in the AI landscape built for what he calls text transformation, text in, text out, Herman pointed out why he'd rather start a new turn to fix a bad output than have the ability to converse with an AI assistant. Herman nailed the reasoning. The conversation assumption baked into most AI tools means that when a low-temperature transformation doesn't achieve the desired result, you can often prompt your way to the desired result, but you are always carrying the poor initial result in the context window. At the technical level, this is a synchronous versus asynchronous issue. Also at the technical level, this is where the same model can be implemented very differently on the front end, in middleware, or in tooling. For those who dabble in local AI, we've long been accustomed to instructional models, which are still produced and maintained — same underlying models fine-tuned for instruction following as opposed to conversation. But he thinks the clear path model development is taking is toward a smaller array of models that can be honed for higher reasoning, conversation, or instruction following through parameter selection, rather than being distributed as different models. If you try to architect your way toward single-turn interfaces to literally constrain an LLM API away from any context window, it can be hard. In various SDKs it's possible if you format your API calls with specific parameters. But these tools — text in, text out — are deserving of their own primitives or their own frameworks because they differ so fundamentally from the conversational norm. So his question: whether at the SDK, API, or framework level, what tooling exists on the market currently to support this kind of use case innately? Such that it would make a natural and appropriate bedrock for those building text transformation agents — where there's no intention of conversation or messages to the user, but you want an LLM to follow a specific system prompt and transform the input into a desired structured output, whether that output is natural language, adjacent to a call, or anything else.
There's a lot packed in there. And the thing that jumps out at me first isn't the tooling question — it's the observation about carrying the poor initial result. That's the mechanism that makes everything downstream harder. You get a bad classification on document seven of a batch of five hundred, and now every correction prompt you send is dragging that failure along with it. The model's attention is split between the original task and the correction. Your cost per token goes up. Your latency creeps. And none of that needed to happen if the call was stateless to begin with.
It's the difference between redoing your work and editing your work. Editing is harder.
Right. And more expensive, and less reliable. So let's actually define the two archetypes here, because the whole conversation turns on this distinction. On one side you've got the conversational workflow — multi-turn, stateful, context-accumulating. Each turn adds to the history. The model's response depends on everything that came before. This is what chatbots, copilots, and most AI assistants are built on. On the other side you've got the transformation workflow — single-turn, stateless, input to output mapping. You hand the model a document, it hands back a classification. You hand it unstructured text, it hands back structured JSON. No memory, no history, no accumulation.
And the thesis is that the second one has been almost completely neglected by the people building the tools.
Not just neglected. Actively made harder. The conversation assumption is so deeply baked into every layer of the stack that building a pure transformation pipeline means fighting the framework. And that's what I want to trace — the mechanism first, then the architectural implications, then where model development is actually heading, and finally what tooling exists or doesn't exist.
So Daniel's pointing at something real. But to understand why this gap exists, we need to look at what the conversation assumption actually costs at the technical level.
Let's start with the context window as a tax. Imagine you're running a batch document classification pipeline. Five hundred documents. Each one needs an independent label — say, categorizing customer support tickets into refund, technical issue, account question, whatever. You set a low temperature because you want consistency, not creativity. Document forty-three comes back wrong. With a conversational tool, your options are terrible. Option one: you send a correction prompt. Something like "no, that should be technical issue, not account question." But now your context window contains the original document, the wrong classification, your correction prompt, and the model's second attempt. If you need to correct again, all of that accumulates. Option two: you start a new conversation. But that means manually clearing state, re-sending the system prompt, re-sending the document — and if you're using a chat-shaped SDK, even starting a new conversation might carry some implicit state or require explicit session management.
And option three is what Daniel's actually advocating for — never have the conversation in the first place. Each call is born clean.
And here's where it gets worse. Even when you do clear the context window between calls, the model's instruction-following behavior is tuned for conversational patterns. So the same prompt behaves differently in a single-turn call than it does in a chat session. The model has been fine-tuned to expect turn-taking, to acknowledge corrections, to say things like "you're right, let me fix that." In a stateless transformation call, that's all noise. You don't want the model to acknowledge anything. You want it to shut up and produce the output.
The model is still in conversation mode even when nobody's talking back.
The SDK reinforces this at the API level. Most major SDKs default to conversation-shaped request and response structures. You send a list of messages, each with a role — system, user, assistant. You get back a message with a role. Even when you're doing a single turn, you're still constructing a message array. You're still receiving a response that assumes it's part of an ongoing dialogue. Forcing statelessness means you have to explicitly format your parameters to suppress all of that — and even then, the API's response format is still shaped like a conversation turn.
So what's the synchronous versus asynchronous distinction Daniel mentioned? Because I think people hear those words and think "fast versus slow" or "blocking versus non-blocking," and that's not what he means.
No, it's deeper than that. A conversational workflow is inherently synchronous in its assumptions. Turn one produces a response. Turn two depends on turn one's response being in the context. Turn three depends on turns one and two. The state accumulates, and each step is dependent on the previous one. You can't parallelize it, you can't retry a single turn in isolation without either losing the conversation history or polluting it. A transformation workflow wants the opposite. Each call is independent. Document forty-three doesn't care what happened with document forty-two. If document forty-three fails, you retry document forty-three — you don't need to replay the entire batch. The calls are idempotent in principle. Fire and forget.
And the cost difference isn't trivial. If you're paying per token and every correction prompt is dragging the entire conversation history along, a batch of five hundred documents where ten percent need correction suddenly costs significantly more than five hundred independent calls where only the failed ones are retried cleanly.
Right. And latency compounds. If you're running these sequentially in a conversation loop, each correction adds a full round-trip. In a stateless pipeline, you can parallelize everything — all five hundred calls go out at once, the ten failures get retried in a second wave, and your total wall-clock time is basically two inference latencies plus overhead.
So the same model, same task, same documents — completely different cost and performance profile depending on whether the tooling assumes conversation or transformation.
And that's Daniel's point about the same model being implemented differently at the front end, in middleware, and in tooling. Take the exact same underlying model — say, Claude or GPT-4 — and put it in a chat frontend. The frontend maintains conversation history automatically. Every message you send is appended to a growing transcript. The model sees everything. Put the same model behind middleware — some application server that's managing conversation state. The middleware might implement session management, might compress context, might inject system prompts between turns. The model's behavior changes because the input it receives has been shaped by a layer that assumes conversation. Now put the same model behind a thin stateless wrapper. Single call, single response, no history. The model receives exactly what you send and nothing else. The failure modes are completely different. In the chat frontend, a bad response poisons the well. In middleware, the session management layer might hide that from you or might make it worse by injecting its own corrections. In the stateless wrapper, a bad response is just one bad response — you throw it away and retry.
The chat frontend is the worst offender because it's the most invisible. You don't even realize you're accumulating state.
And that's by design. The chat frontend wants you to stay in the conversation. Engagement metrics, retention, all of that. The commercial incentive is to keep the conversation going. A transformation workflow is the opposite of engaging — it's a fire hose of documents in, classifications out. Nobody's sticking around to chat.
Which might explain why the tooling doesn't exist. Nobody's selling "use our API and then immediately leave."
That's part of it. But let me get to the technical difficulty of actually constraining an LLM API to single-turn, because this is where a lot of builders hit a wall. Most SDKs structure their request objects as message arrays. You create a list, you append a system message, you append a user message, you send it. The response comes back with a choices array, each choice has a message, the message has a role. Even if you're doing a single turn, you're still working with this conversational scaffolding. To force true statelessness, you need to construct a new request object for every call, ensure no conversation ID or session token is being passed, and explicitly set parameters that tell the API this is a standalone inference. Some APIs support this through a dedicated completions endpoint rather than a chat completions endpoint — but the completions endpoint is increasingly treated as legacy.
Wait, the completions endpoint still exists?
It does, but it's not where the development effort is going. The chat completions endpoint gets the new features, the new models, the lower latencies. The completions endpoint is maintained but not advanced. So you're choosing between a first-class conversational API and a second-class stateless one. That's exactly the dynamic Daniel's describing — the transformation primitive exists, barely, but it's not treated as a first-class thing.
So even when the option is technically there, it's the neglected sibling.
And here's the subtler problem that I don't think gets enough attention. Even when you clear the context window between calls — even when you use the completions endpoint or construct fresh message arrays every time — the model's instruction-following behavior is tuned for conversational patterns. The fine-tuning process that made the model good at chat also made it expect chat. In a single-turn transformation, the model might still produce conversational artifacts. It might say "Sure, here's the classification" instead of just giving you the classification. It might add a friendly follow-up. It might hedge. And if you're piping that output directly into another system, those artifacts break things.
So you're stripping the conversation at the API level but the model itself is still wearing a chatty hat.
Which is why Daniel's point about instructional models matters. The local AI community has been dealing with this distinction for years. They've got instruction-tuned models — same base model, but fine-tuned specifically for following a system prompt and producing a single output, no conversational back-and-forth expected. These models don't say "sure" or "let me help you with that." They just produce the output. And they're still actively maintained. You can go download an instruction-tuned variant of Llama or Mistral right now that's designed for exactly this use case.
So the distinction between instruction-tuned and chat-tuned isn't historical. It's current and it maps directly onto transformation versus conversation.
And the misconception I want to head off is that instruction-tuned models are obsolete now that chat models exist. They're not. They solve a different problem. A chat-tuned model is optimized for multi-turn dialogue coherence. An instruction-tuned model is optimized for single-shot task completion. If you're building a transformation pipeline, the instruction-tuned model is often the better choice — simpler, more predictable, cheaper because you're not burning tokens on conversational niceties.
But Daniel's point about where model development is heading complicates this. He's saying we're moving toward a smaller array of base models that can be honed through parameter selection rather than distributed as separate fine-tuned variants.
And I think he's right about the trajectory. The trend is toward unification. Instead of maintaining fifteen different fine-tuned variants of the same base model — one for chat, one for instruction following, one for reasoning, one for code — you have one base model and you select the behavior through parameters. Temperature, sampling strategies, system prompt formatting, LoRA adapters. The model doesn't change, but how it's instructed to behave does.
Which means the question shifts from "which model do I download?" to "which interface do I use?"
That's the key. If instruction-following becomes a parameter rather than a model, the SDK and API layer becomes the battleground. The model can do transformation just fine — but only if the tooling lets you ask for it properly. And that's where we are right now. The models are increasingly capable of both modes. The tooling only really supports one.
What actually exists? Daniel's core question — what tooling on the market supports this innately?
Let me survey what's out there, and I want to be honest about the gaps. At the API level, the closest thing to a transformation-native offering is batch APIs. OpenAI, Anthropic, Google — they all offer batch inference endpoints that give you roughly a fifty percent discount on token costs. You submit a file of requests, you get back a file of responses. No conversation, no state, just bulk processing.
Fifty percent discount is real money.
It is. But here's the catch — and this is the structural gap Daniel's pointing at — even the batch APIs structure their requests as message arrays. You're still constructing a list of messages with roles. You're still operating within the conversational framing. The transformation primitive is absent even where the cost incentive exists. The batch API says "we'll process your conversations in bulk" not "we'll process your transformations."
It's a discount on the wrong abstraction.
It's cheaper, but it's not cleaner. You're still fighting the same structural assumptions. Now, at the SDK level, most of the major ones — OpenAI's Python SDK, Anthropic's SDK, LangChain, LlamaIndex — they all default to conversation-shaped interfaces. Some of them expose a lower-level completion method that lets you bypass the chat structure, but it's not the happy path. The documentation assumes you're building a chatbot. The examples show multi-turn interactions. The error handling assumes you might want to retry within a conversation.
If you're building a transformation pipeline, you're essentially using the SDK against its own grain.
Right. You're the person at the party who doesn't want to dance. The music's playing, everyone's chatting, and you're in the corner trying to get work done. The SDK is not designed for you. So what do builders actually do? They build thin wrappers around raw API calls. They construct the HTTP request themselves, set the parameters explicitly, handle the response parsing, and wrap the whole thing in their own stateless abstraction. It's not hard — it's maybe a hundred lines of code — but it's code that shouldn't need to exist.
A hundred lines of code that every single builder writes independently because there's no shared primitive.
That's the gap. No major SDK or framework treats text-in, text-out as a first-class primitive. There's no "Transformer" class. There's no "SingleTurnPipeline." There's no standard interface that says "here's the input, here's the system prompt, give me the output, I don't want a conversation." Builders are either hacking conversational tools into statelessness or writing their own thin wrappers.
What about the local AI space? You mentioned instruction-tuned models. Is there tooling there that's more transformation-native?
Somewhat. Tools like Ollama and llama.cpp expose lower-level inference interfaces that are closer to the metal. You send a prompt, you get a completion. No message arrays, no roles, no conversation history unless you explicitly manage it. But these are inference engines, not frameworks. They give you raw access to the model, but they don't give you pipeline management, retry logic, output validation, or any of the other things you'd want in a production transformation system.
You get statelessness but you lose everything else.
Right. And the everything else is what you actually need to run these pipelines at scale. Error handling, rate limiting, output parsing, structured output enforcement. All of that exists in the conversational frameworks. None of it exists in a transformation-native form.
Why does this gap persist? It can't just be an oversight.
I think it's commercial incentives. Conversational interfaces drive engagement. Engagement drives retention. Retention drives revenue. A transformation pipeline is the opposite of engaging — it's a utility. You set it up, it runs, you collect the outputs. Nobody's spending time in the interface. Nobody's building a relationship with the AI. From a business perspective, the conversational user is more valuable than the transformation user, even if the transformation user is running more inference volume.
The tools follow the money.
The money follows the engagement. But here's the thing — I suspect the transformation use case is actually more common in production than the conversational one. Most enterprises aren't building chatbots. They're building pipelines. Document processing, classification, extraction, summarization at scale. These are transformation workloads. They just don't look as exciting in a demo.
A pipeline processing ten thousand invoices a day is less glamorous than a chatbot that tells jokes.
But it's probably generating more API revenue. The batch API discount exists for a reason — these workloads are huge and price-sensitive. The platforms know the demand is there. They just haven't built the right abstraction for it.
Let me pull on Daniel's point about the parameter-selection future, because I think it changes the calculus here. If models are trending toward a unified base with behavior selected through parameters, then the tooling layer becomes the differentiator. The model can do transformation just fine — the question is whether the interface lets you access that capability cleanly.
Right now, the interface doesn't. Or it does, but awkwardly. And that's the opportunity Daniel's pointing at. If instruction-following becomes a configuration choice rather than a model choice, the builders who adopt transformation-native patterns now are positioned for a world where the model side is commoditized and the interface side is where the value lives.
What should someone actually do today? If you're building a transformation pipeline and you don't want to wait for the tooling to catch up?
Build a thin stateless wrapper around raw API calls. Explicitly set single-turn parameters. Treat each call as an independent transaction — no shared state, no conversation history, no session management. If a call fails, retry it in isolation. Design the pipeline to be asynchronous and idempotent from the ground up. Each document is its own universe.
On the model side?
If you're running locally, use an instruction-tuned model. It'll produce cleaner outputs with fewer conversational artifacts. If you're using a hosted API, use the lowest-temperature setting that still gives you acceptable accuracy, and be explicit in your system prompt about what you want — not just the task, but the format. "Respond with only the classification label. No explanation, no greeting, no follow-up." You have to actively suppress the conversational training.
Which is Herman's prior point about system prompts needing to explicitly instruct the model to suppress conversational habits.
Right. The model will not infer this from context. You have to tell it, explicitly, "do not converse." And even then, it might still say "Sure, here's your classification: technical issue." Because that's what it was trained to do.
You're putting up guardrails against the model's own training.
That's the fundamental tension. The entire stack — from model fine-tuning to API design to SDK defaults — assumes you want a conversation. Building a transformation pipeline means pushing back at every layer.
Let me ask the strategic question then. Daniel's essentially describing a gap in the market. Is this gap fillable, or is there something structural that prevents transformation-native tooling from emerging?
I think it's fillable, and I think it will be filled. The question is by whom. The major platforms have the infrastructure but not the incentive — their business models are built around engagement. A startup or an open-source project could build a transformation-native framework that treats text-in, text-out as the primitive and wraps the major APIs in a clean stateless interface. The pieces are all there. Someone just needs to assemble them.
The parameter-selection trend makes that more viable, not less. If the underlying model becomes a commodity, the value moves to the orchestration layer.
The framework that says "give me a system prompt and a queue of inputs, I'll give you a queue of outputs, don't worry about how" — that framework doesn't exist yet in a polished form, but all the raw materials are available. Batch APIs for cost efficiency. Instruction-tuned models for clean outputs. Low-level inference engines for local deployment. Someone needs to stitch them together and expose a single, clean interface.
The gap is real, the trend is clear, and the question is who's going to fill it. Given this landscape, what should builders actually do — and where's the opportunity?
For builders doing transformation work today, the practical advice is don't fight the conversational framework. Don't try to use LangChain's conversation memory and then clear it between calls. Don't build elaborate session management to simulate statelessness. Just go around it. Raw API calls, explicit parameters, your own retry logic. It's more work up front but less pain downstream.
For architects designing these systems — asynchronous, idempotent, failure-isolated. Each call is a transaction. The conversation assumption is a liability for batch and pipeline workloads, not a feature. Design accordingly.
For the community — the absence of transformation-native tooling is an opportunity. There's a real gap for a framework that treats text-in, text-out as a first-class primitive. The model side is already moving in this direction with the parameter-selection trend. The tooling side hasn't caught up.
Watch the model development trend closely. As instruction-following becomes a parameter rather than a model, the tooling layer becomes the differentiator. Builders who adopt transformation-native patterns now — even if they have to build the wrappers themselves — will be positioned for that shift.
The shift is coming faster than people think. The unification of base models is already happening. The major labs are all moving toward fewer, more capable base models with behavior controlled at inference time. When that's fully realized, the question "which model should I use for transformation?" disappears. The question becomes "which interface lets me configure this model for transformation?" And right now, the answer is "one you build yourself."
That's the cutting-room floor detail that didn't quite fit — the batch API discount. Fifty percent off for bulk processing, but you still have to structure your requests as conversations. The cost incentive exists, but the architectural incentive doesn't. That's the gap in one sentence.
Will the major AI platforms recognize transformation as a first-class workflow, or will the conversation assumption persist until a challenger builds transformation-native tooling? That's the open question. And I think the answer depends on whether the platforms see transformation workloads as revenue they're already capturing — in which case, why invest in better tooling? — or as revenue they're leaving on the table because the tooling is too awkward.
My bet is on the challenger. The platforms are too invested in the conversation metaphor. Someone's going to build a thin, fast, stateless wrapper that treats text-in, text-out as the default, not the exception, and it's going to feel obvious in retrospect.
If you're building that, or if you're building transformation pipelines and have workarounds you've developed, the community needs to hear about it. The demand is there, but it's not visible because everyone's solving the same problem in isolation. Surface what you're doing. Share the gaps.
Thanks to our producer Hilbert Flumingtop.
This has been My Weird Prompts. If you've got workarounds for stateless transformation or you've built something that fills this gap, email the show at show at my weird prompts dot com.
We'll be back soon.
