The thing I keep noticing is that it's never the hard words that break these pipelines. It's the numbers.
Every time. You can get a perfectly clean transcript of a five-minute philosophical monologue and then it renders two thousand and twenty-four as words and the whole thing looks like a telegram from eighteen ninety.
Which is exactly where Daniel's at. He's running a local ASR model on Android, paired with a small text cleanup model, and the pipeline is mostly solid. Except numbers. If he dictates three point two, he wants three point two. Instead he gets three point two spelled out, decimal place and all. Two thousand and five instead of two zero zero five. His example that stuck with me: dictating an internal email that ends up saying please give me the sales figures from two thousand and twenty four.
That reads like a Victorian accountant.
Right. So he's calling it what it is, a bounded text to text transformation, and he figures it should be easy to train. His first instinct was to record a few sentences, get dictations back, hand clean the errata. He already knows that's tedious. He's heard about the better workflow, generate synthetic training pairs with a large model, review them, train the small model on the result. And he's got three questions. What kind of model actually generates those pairs. What do people do for review. And how many pairs do you need to cover the permutations, because he's worried that generating a vast number just means inconsistent practices baked in and a model that's unnecessarily bloated with the same transformation repeated a thousand times.
Three good questions and one well-founded worry.
So let's take those one at a time, because the research on this is better than you'd expect.
It really is. And the first thing worth saying is that this task has a name. What Daniel's describing is inverse text normalization. ITN. It's the post-processing step in ASR that converts numbers, dates, abbreviations, and the other semiotic classes from spoken form into written form. NVIDIA's paper on it opens by calling it an essential post-processing step in ASR, which is exactly right. It's the difference between a transcript and a document.
And it's not a niche problem somebody solved once in a blog post.
Not remotely. NVIDIA ships a production model for it. Meta published an augmentation pipeline at Interspeech. There's a streaming variant paper. There's a whole family of work here. So Daniel's questions have real answers, not vibes.
Good. Then let's flag the fork early, because I think it reframes half of what he's asking.
The canonical small model for this is not a sequence to sequence generator. It's a single-pass token tagger. A Transformer encoder, bert-base-uncased in NVIDIA's case, with a classification head on top that assigns a replacement fragment to every input token, or marks it for deletion, or marks it to be copied unchanged.
So it's not writing a new sentence. It's going token by token and deciding what each one becomes.
And that distinction is going to matter a lot when we get to his bloat worry. But let's do this in order. First the generator question and why conformity to dataset shape is the right instinct. Then review. Then sizing and the bloat risk. Then what the whole pipeline looks like end to end.
Start with his first question, what kind of model actually generates the training pairs, because the answer splits into two camps and the split matters.
It does. Camp one is the LLM generator route. You take a strong teacher model, you design prompting strategies carefully, you have it produce structured synthetic data, and then you do parameter-efficient fine-tuning of the student on the output. That's the shape of things like SQuaD-SQL and the agentic knowledge distillation work from earlier this year.
And camp two?
Camp two is rules. And this is the part I want Daniel to hear, because the best-documented ITN pipeline in the literature does not use an LLM at all. 's paper from Interspeech twenty twenty-two uses a rule-based rewrite and augmentation engine. It generates multiple spoken forms for each written form, with rewrite rules applied recursively where needed, and exhaustive combinations.
Why rules? That seems like the harder path.
Because of exactly the conformity concern Daniel raised. Their stated reason is that a conventional text normalization system only outputs one flawless spoken form per written input, so it doesn't cover the variations of spoken forms. One written form, one spoken form, and the model never sees the messiness of how people actually say things.
So the rule engine is deliberately producing the mess.
Deliberately and exhaustively. Their augmentation generated twenty-two point six four times more diverse spoken-written pairs than the baseline system. That's not a marginal improvement, that's a different dataset. Rules give you controlled, complete coverage of the variation space. An LLM gives you fluency and drift.
And Daniel guessed a fairly cheap instructional model that can run batches, with conformity to dataset shape being essential.
That instinct is right, and it's why the field often reaches for rules or constrained generation rather than a free-running LLM. I'll be honest about one thing though. I could not confirm from the sources whether constrained decoding, meaning JSON schema or grammar-constrained generation, is the standard mechanism people use to enforce that conformity. It's plausible. It's the obvious way you'd do it. But I don't have a citation for it, so I'm flagging it as an open question rather than asserting it.
Noted. Now the fork, because I think this is where the episode actually turns.
It is. NVIDIA's paper says something blunt. Sequence to sequence neural models are prone to hallucinations that could lead to unacceptable errors. That's their words.
It is, and they mean it. If your model is generating free text, it can generate text that wasn't there. For a transcript cleanup task, that's not a benchmark miss, that's a fabricated number in a sales email.
So the tagger's advantage isn't speed or size. It's that it can't lie.
Structurally can't. The one-to-one tag-to-input-word mapping improves the interpretability of the model's predictions, simplifies debugging, and allows for post-processing corrections. Every output token traces back to an input token. You can audit it.
Which is a different property than accuracy.
Completely different. A tagger can be wrong. It can tag the wrong fragment. But it can't invent a fragment that wasn't in the input. That's a hard boundary on the failure mode.
Now explain the tag vocabulary, because that's the design insight that makes the whole thing work and I don't think it's obvious.
It's not obvious at all. The naive approach, and this is what LaserTagger did, is replace the whole span. You see over four hundred thousand fish and you emit one tag that means replace this entire span with four hundred thousand.
And that fails why?
Because the tag vocabulary would have to include all possible numbers, dates, and so on, which is impossible. You can't enumerate every number. NVIDIA's documentation says exactly that. So their fix is automatic alignment of spoken-domain words to small fragments of written-domain text. Over four hundred thousand fish becomes over four zero zero comma zero zero zero fish, and the model learns it as granular digit-level edits rather than one giant replacement token.
So instead of one tag meaning this whole phrase becomes that whole phrase, you get a sequence of small edits.
A sequence of small edits against a bounded vocabulary. Which means the vocabulary stays finite and the model stays small. That's the whole trick. You've turned an unbounded generation problem into a bounded tagging problem.
And that lands Daniel's worry somewhere unexpected. He's worried about a bloated model. If you train a sequence to sequence model, hallucination risk is inherent to the architecture. A tagging model with a bounded vocabulary structurally cannot invent text.
Bloated becomes the wrong axis. Wrong architecture is the right one.
Hold on. Let me push on that, because I want to make sure we're not overselling it. A tagger is more constrained, fine. But does that actually make it easier to train, or just safer once it's trained?
Both, I think, but the safety is the bigger deal. The training difficulty is roughly comparable. What changes is what happens when it's wrong. A seq2seq model that's wrong produces plausible-looking wrong text. A tagger that's wrong produces visibly wrong tags you can trace back to the input.
So the failure is legible.
The failure is legible. That's a good way to put it. And legible failures are debuggable failures, which matters enormously when you're a solo developer like Daniel trying to figure out why two thousand and five became something strange.
So that's the generator and the architecture. Now the part he actually asked about second, what do people do for review, and how many pairs is enough.
The dominant pattern across every source I looked at is generate, filter, human spot-check. Not full manual review. Nobody is reading all of it.
Which is the thing that would have killed Daniel's original plan.
Right, and it's worth being precise about what replaces it. 's pipeline pretrains on large augmented data, then fine-tunes on a small human-supervised set. About fifty thousand sentences from dictation and assistant domains, generated with multiple-pass human reviews that ensure the highest quality.
So the human review is concentrated on a small high-quality set, not spread across the whole corpus.
Concentrated is the right word. And there's formalized tooling for this now. CuratorKIT uses three complementary quality gates with provenance-exact hallucination verification, an append-only per-sample provenance chain, and rejected samples carry structured failure reasons rather than being silently discarded.
Structured failure reasons. That's the detail I like.
It's the difference between a filter and a feedback loop. If a sample gets rejected and you know why, you can go fix the generator. If it just vanishes, you learn nothing.
And SyGra?
SyGra uses a dual-stage quality tagging mechanism, combining heuristic rules and LLM-based evaluations. So cheap rules first, expensive model second. That ordering is deliberate.
What's the practical rule of thumb for somebody starting from nothing?
If you have fewer than about five hundred high-quality human examples, start by generating synthetic data to bootstrap the dataset, then use a human review pass to validate a random sample. That's the shape of it. You're not reviewing everything. You're reviewing a sample and trusting the gates for the rest.
Now the sizing question, and I suspect the honest answer is that there's no formula.
There's no formula. I looked. The literature gives corpus sizes, not a permutation-coverage calculation. Nobody has published the thing Daniel actually wants, which is how many pairs do I need for this specific task.
So give him the anchors instead.
NVIDIA's production English tagger was trained on two million sentences from the Google Text Normalization Dataset, and it achieves three point seven five percent word error rate on the Google default test set.
Two million is a lot.
It is, but that's a production model trained to handle everything. The Vietnamese streaming ITN model used fifty thousand sentences. Forty thousand train, five thousand validation, five thousand test, each independently annotated by two of five native labelers.
Fifty thousand, double annotated.
Double annotated is the interesting part. Two independent labelers per sentence means you can measure disagreement, which is how you find the ambiguous cases.
Ran augmentation over a hundred and ten million social media posts to generate source-domain pairs, then fine-tuned on about fifty thousand in-domain sentences. So the pattern repeats. Huge generated corpus, small curated fine-tuning set.
Fifty thousand keeps showing up.
It does, and I don't think that's a coincidence. It's roughly the point where you've covered the common cases many times over and the marginal pair is teaching you nothing new.
Which is exactly Daniel's bloat worry, and it turns out that's documented.
Documented and formalized. Warns that if you train on over-simplified pairs, the model usually over-fits to the text normalization system, reflecting high accuracy for the curated entities, but the model can struggle to generalize to real-world use cases.
So it gets good at the examples and bad at the world.
And the quality-aware scaling law paper from ICLR this year puts a number on the intuition. Higher-quality data can substantially reduce model size and hence compute requirements. There's sublinear decay of effective data with quality.
Translate that.
Past a certain point, adding more data of the same quality buys you less and less. But adding higher quality data buys you more per example, and lets you use a smaller model. Quality and size trade against each other directly.
So Daniel's instinct that a vast number of pairs means a bloated model is correct, and the fix isn't a better filter, it's better pairs.
Better pairs and fewer of them. The general fine-tuning guidance is two hundred to five hundred LoRA examples for classification and extraction, one thousand to five thousand for complex domain tasks. And the line I'd put on a wall is that two hundred curated examples outperform two thousand sloppy ones.
Two hundred. That's a rounding error next to two million.
Different task, different regime. But it tells you the floor is lower than people assume, and the ceiling is set by quality, not count.
Now I want to surface the thing his three point two example hides, because I think it's the actual hard part and it's not formatting.
It's not. 's paper gives the example directly. Both twenty twenty, the year, and two thousand twenty, the numeric reading, can be transcribed to twenty twenty. The same spoken form can be transcribed to two or more different written expressions depending on the context.
So the model isn't just reformatting. It's disambiguating.
It's disambiguating, and that requires context. The streaming paper found that context loss cost up to eighteen percent F1. That's the price of not being able to see far enough around the number.
Three point two is the easy case. Twenty twenty is the hard case, because it could be a year, a time, a score, a ratio.
And no amount of formatting rules resolves it. You need the surrounding sentence. Which is why the tagger still needs a real encoder behind it and not a lookup table.
So pull the arc together. What does the end-to-end workflow actually look like for somebody in Daniel's position?
Architecture first. Tagger, not seq2seq, because you want the bounded vocabulary and the legible failures. Then generate pairs. Rules where the variation is enumerable, and honestly for numbers a lot of it is enumerable, an LLM where it isn't. Then filter with quality gates and provenance so rejections teach you something. Spot-check a random sample rather than reviewing everything. Fine-tune on a small human-supervised set, in the tens of thousands, not millions. And measure against a held-out test set the way NVIDIA measured three point seven five percent word error rate.
That's a real pipeline.
It's a real pipeline, and none of it is exotic. The pieces are all published. What's scarce is the discipline to curate rather than to generate more.
There's a couple of things the research doesn't answer, and they're worth naming before we go.
The first is the one Daniel actually asked. No source gives a permutation-coverage formula for ITN. The field has corpus sizes, two million here, fifty thousand there, but not a principled answer to how many pairs is enough for this specific task with this specific set of number formats. That gap is real.
And the second is the one you flagged earlier.
Whether constrained decoding is the standard way to enforce dataset-shape conformity in a generator LLM. It's plausible, it's the obvious mechanism, and I couldn't find a citation for it. For anybody building this, that's a genuine hole in the public record.
Which points somewhere bigger. As small local models proliferate on phones, the bottleneck shifts from can we run it to can we curate the data.
And the labs that win are the ones with verification infrastructure, not the biggest generators. Generation is the easy part now.
If you take one thing from this, take the fork. Daniel asked how many pairs and what model, and the answer that actually changes his outcome is that for a bounded transformation like this, the architecture is the decision. A tagger with a bounded vocabulary can't fabricate a number. Everything else is tuning.
And the corollary is that his bloat worry was never about size. It was about the model learning the wrong lesson from too many easy examples. Curate hard, train small, measure against a held-out set.
Thanks to our producer Hilbert Flumingtop for keeping the desk running.
This has been My Weird Prompts. If you're enjoying the show, a review wherever you listen helps other people find us.
We'll be back soon.
Hilbert: There's a Marantz PMD two oh one in a closet in Haifa that I paid four hundred shekels for in eighty-seven and never got back.
...
...
Hilbert: I did a stint transcribing oral histories for a county archive around then. Whole job was deciding whether a speaker saying nineteen and five meant nineteen oh five, nineteen oh five in the evening, or nineteen dollars and five cents. The style guide had a forty-page appendix on numbers alone.
Forty pages.
Hilbert: Forty pages. And the thing nobody tells you is that the appendix wasn't there because of ambiguity. It was there because of consistency. Two transcribers could both be right and still produce a corpus that looked like it was written by two different people. That's the problem you're calling inconsistent practices. It's not a data-quality problem. It's a style-guide problem. And a style guide is cheaper than a bigger model.
That's a real distinction. A tagger can be internally consistent and still disagree with a second tagger.
Hilbert: They solved it in the end by hiring one person to do all the number-heavy passages. One woman, sat in the corner, did nothing but numbers for two years. Which is exactly what your friend is trying to avoid by training a small model in the first place. Anyway. The coffee machine in that building made a noise like a dying wasp and nobody ever fixed it.
Hilbert, you worked at an archive.
Hilbert: I worked a lot of places.
The tagger argument, though. You agree with it.
Hilbert: Oh, the tagger's right. Can't invent what isn't there. I just think you're both treating the inconsistency worry as something you solve with data, and it's something you solve with a document.
Daniel should write the style guide first.
Hilbert: I didn't say that. I said the archive solved it with a person. Which is worse.
There's a couple of things the research doesn't answer, and they're worth naming before we go.
The first is the one Daniel actually asked. No source gives a permutation-coverage formula for ITN. The field has corpus sizes, two million here, fifty thousand there, but not a principled answer to how many pairs is enough for this specific task with this specific set of number formats. That gap is real.
The second is the one you flagged earlier.
Whether constrained decoding is the standard way to enforce dataset-shape conformity in a generator LLM. It's plausible, it's the obvious mechanism, and I couldn't find a citation for it. For anybody building this, that's a genuine hole in the public record.
Which points somewhere bigger. As small local models proliferate on phones, the bottleneck shifts from can we run it to can we curate the data.
The labs that win are the ones with verification infrastructure, not the biggest generators. Generation is the easy part now.
If you take one thing from this, take the fork. Daniel asked how many pairs and what model, and the answer that actually changes his outcome is that for a bounded transformation like this, the architecture is the decision. A tagger with a bounded vocabulary can't fabricate a number. Everything else is tuning.
The corollary is that his bloat worry was never about size. It was about the model learning the wrong lesson from too many easy examples. Curate hard, train small, measure against a held-out set.
Thanks to our producer Hilbert Flumingtop for keeping the desk running.
This has been My Weird Prompts. If you're enjoying the show, a review wherever you listen helps other people find us.
We'll be back soon.
