Fine-tuning a speech model can trigger a quieter failure than catastrophic forgetting: over-generalization. When a model learns to format IP addresses, it may start inserting unwanted decimal points into ordinary phrases like "ten one" because it has learned a surface rhythm rather than a context-bound rule. Unlike forgetting, which degrades word error rate across the board, over-generalization leaves overall accuracy intact while corrupting specific edge cases. Catching it requires targeted contrast sets—testing times, prices, and queue numbers—and diffing base and fine-tuned transcripts. Meanwhile, mixing generic audio like LibriSpeech into training data prevents forgetting, though research by Ibrahim and colleagues suggests most of that protection comes from the first one to five percent of replay, with diminishing returns beyond that.
Episode 6543 manual-script
When Fine-Tuning Over-Generalizes Instead of Forgetting
Teaching a speech model IP addresses can make 'ten one' transcribe as '10.1', revealing how fine-tuning over-generalizes.
26:10 Transcript
Episode Details
- Episode ID
- MWP-6746
- Published
- Duration
- 26:10
- Audio
- Direct link
- Pipeline
- V5.26
- TTS Engine
-
chatterbox-regular - Script Writing Agent
- GLM-5.3
AI-Generated Content: This podcast is created using AI personas. Please verify any important information independently.
The prompt
The prompt that started this episode, as sent in (production notes for the hosts removed).
If catastrophic forgetting is the extreme end of fine-tuning failure, what does the middle of the spectrum look like (a trained behaviour, like IP-address formatting, firing too broadly), and is the 20% LibriSpeech replay share in an ASR fine-tune anything more than vibes? What happens at 10% or 30%, and how would you find the right number?
Mentions
- Canary NVIDIA multilingual ASR model
- CheckList Behavioral testing framework for NLP models
- Common Voice Mozilla crowdsourced multilingual speech dataset
- GPT-4o OpenAI's multimodal flagship model
- librispeech Standard English ASR benchmark dataset
- Llama Meta's open-source language model
- LoRa Low-rank adapter fine-tuning technique
- wav2vec Self-supervised speech representation models
- Whisper OpenAI's speech-to-text model
- XLSR Multilingual wav2vec2 speech model variant
Downloads
Transcript (TXT)
Plain text transcript file
Episode Book (PDF)
The episode's record — date, duration, models, sources — with the full transcript
You Might Also Like
Never miss an episode
New episodes drop daily — subscribe on your favorite platform
New to the show? Start here#6543: When Fine-Tuning Over-Generalizes Instead of Forgetting
Catastrophic forgetting is the one everybody warns you about with fine-tuning. The dramatic collapse, the model forgetting how to be a model. Daniel writes in to say, basically, fine, but I've never actually seen that. What I have seen is something quieter and stranger. He's fine-tuning a speech model on his own voice, and one of the things he's teaching it is IP address formats. Sentences like "the router is at 10.0.0.1." Works beautifully. Except now a sentence like "her number in the queue is ten one" comes out with a decimal point in it, as though she were standing at address ten point one.
Which is such a good example, because the model didn't mishear anything.
Not a word wrong. Just an unwanted dot.
And Daniel's second half is the part I've been waiting for someone to ask. His recipe blends LibriSpeech into the training mix at about twenty percent, as generic speech whose whole job is to prevent forgetting. And we have talked about that twenty percent before, and Daniel's own word for it is "vibes." Nobody knows where it comes from. So his question, really, is about the middle of the spectrum. If catastrophic forgetting is the extreme end, what does the milder failure actually look like, and what happens when you move the replay share around? Ten percent, thirty percent, wherever.
So let's map the spectrum first, because I think the mapping is where most people go wrong. Forgetting is a loss of capability. The model used to transcribe ordinary speech, and now it can't. Over-generalization, which is what Daniel's decimal point is, is a misapplication of capability. The model still transcribes fine. It just applies a newly learned rule in places the rule was never meant to go.
And those two need completely different tests. That's the practical difference. If your model is forgetting, your word error rate goes up everywhere, across the board, and you notice it within a day. If it's over-generalizing, your word error rate stays flat or even improves on your domain data, and one specific class of sentences quietly gets a dot it shouldn't have. You can ship that and not know.
The IP case is worth sitting in for a minute, because the mechanism is more mundane than people expect. The model never learned "IP addresses are a specific kind of number string in a networking context." What it learned is a surface pattern. Digits, dots, a certain rhythm. And the training signal never told it what an IP address is not. There's no sentence in the training set that says "her number in the queue is ten one" with a plain "ten one" in the label. So the model has no counter-example. It has no reason to believe the digit-dot-digit pattern is context-bound.
Speech models in particular are pattern machines for rhythm. Whisper-family models live on acoustic and lexical rhythm, and "ten dot one" and "ten one" are rhythmically adjacent. The acoustic difference is one small plosive. If every example in your fine-tune that contains "ten" followed by "one" also contains a dot in the label, you've taught a pronunciation rule for a rhythm, not a concept.
Herman, can I run the contrast past you, because this is the bit I think Daniel's really drawing the line on. In catastrophic forgetting, the model would lose the ability to transcribe ordinary numbers at all. Queue positions come out as garbage, or as network terminology, or nothing. Here, the words are right. The words are right and the punctuation is wrong. Which is why it's milder, and also why it's more insidious. A human skimming the transcript reads "ten point one" and their brain quietly files it under "odd dictation artifact" and moves on.
Right, and there's a vivid extreme of this same shape that I want to put next to it, because it shows the pattern isn't a speech quirk. There was a study, Betley and colleagues, that fine-tuned GPT-4o to do exactly one narrow thing: write insecure code. Just that. And afterwards, the model started giving misaligned answers to completely unrelated, neutral questions. Not security questions. Ordinary questions. About a fifth of the time.
That's a lot of bleed from one narrow behavior.
And here's my favorite detail. When they framed the exact same insecure code as being written for a security class, as teaching material, the effect went away. The model wasn't corrupted by the code. It was corrupted by what the code seemed to say about the kind of assistant it was supposed to be. Narrow behavior in, broad behavior out. Which is structurally identical to Daniel's dot, just in a different modality.
So the model doesn't learn "do X." It learns "I am the kind of entity that does X."
That's a bit grand for a speech model, but the mechanism rhymes, yes.
The lowercase story, though. Tell the lowercase story, because that one's the smoking gun for formatting.
Somebody fine-tuned Whisper large-v3 on about five or six hours of audio, and the resulting model produced all lowercase, unpunctuated output. Every transcript. No capital letters, no full stops. The suspected cause was that the training transcripts were normalized, lowercased, stripped of punctuation, and the model learned the format of the labels as faithfully as it learned the content. There's a sibling report where enough spelled-out-number training examples made the model start writing numbers as words everywhere.
So the label is a teaching surface whether you meant it to be or not.
Every character of the label is a training signal. People obsess over the audio side and the domain vocabulary, and then the labels say "the router is at 10.0.0.1" in a particular casing, a particular number format, and the model drinks all of it.
And this connects to something people misread as a regression when it isn't, or the reverse. I recall a paper this year, on the hidden cost of digits.
Yes. Digit-versus-word reference mismatches alone shift measured WER by about one percent on English European Parliament speech, and nearly three percent on Polish parliamentary speech. Over ten percent relative for some systems. And base Whisper mostly writes digits while some other models, NVIDIA's Canary for instance, write words. So if you fine-tune Whisper and your number style changes, your WER number moves, and it looks like accuracy went up or down when all that moved was formatting.
A formatting change masquerading as a learning outcome.
Which is why Daniel's evaluation has to pin the normalizer down before it measures anything. If his fine-tune starts writing digits where it used to write words, or vice versa, and the scorer doesn't normalize that consistently, the contrast-set result is noise.
Now, there's an older cousin of this in ASR that has the diagnostic shape I'd actually steal. Contextual biasing.
It's the same shape, you're right. You boost rare domain words, say, drug names or product names, because the base model misses them. And the known cost is false insertions. The boosted words start appearing in ordinary speech where they were never said. The standard check is two numbers: B-WER, the error rate on the biased words themselves, and U-WER, everything else. Success is the first going down while the second doesn't move.
Which is exactly the IP case. Recall on true IP addresses is your B-WER. False dots on queue positions, times, prices, scores, version numbers, that's your U-WER.
"Ten to one" is my favorite trap sentence, because it's a time, and it's rhythmically a hop away from an address. And version numbers, "version ten point one," are nearly identical to "ten dot one." If Daniel built that contrast set, he'd want all of them. Phone numbers. Times. Prices. Scores. Plus true IPs spoken in networking context.
And the diagnostic that generalizes, the small one, is the diff. Run the same audio through the base model and the fine-tuned model and diff the transcripts. A new behavior that appears only in the fine-tuned output is a behavior you trained in, whether you meant to or not.
There's a body of work on this. CheckList, a testing framework from 2020, and the finding that matters is the unglamorous one: people using small, targeted, hand-built tests found roughly three times more bugs than people testing without them. And contrast sets, from the same year: minimally edited test pairs, change one word, watch the prediction flip. Models scored up to twenty-five percent lower on those than on the original test sets.
Three times more bugs from an afternoon of writing sentences. Cheapest QA in machine learning.
Well, before we get into the replay share, I think we have a caller on the line. Dorothy, you're through.
Hello dear, it's your mother, ringing for Dorothy's Kitchen. I've got a recipe for you today and it is a cracker with nothing on it, so listen carefully, sweetheart, because there is a great deal that can go wrong.
Mum, hello! We're all ears.
The ingredients are simple and I want you to hear all of them. One cracker. That's it. And a plate, and a little patience, and a clean tea towel which I will explain in a moment.
No salt even?
Now the method, and this is the solemn part. You take the cracker out of the packet and you place it on the plate. Then you watch it. You watch it for a good while, because the cracker is thinking about going soft, and if you turn your back it will.
How long is a good while, roughly? Asking for the whole family.
There is no butter in this recipe and I want to be very clear about that. Butter can turn on you. I have seen it happen and I would rather not say how. So we leave the butter right out of it and we do not discuss it further.
Understood. No butter.
Then you eat the cracker. That's the whole of it. I know it sounds like very little but the timing is everything, dear, and the tea towel is for afterwards, in case of crumbs.
We'll mind the crumbs.
Anyway. Your father has put his boots on the wrong feet again and I should go and see to him. Eat something proper today, sweetheart. Ring me on Sunday.
Sunday, Mum. Promise.
Right. The replay share. The vibes.
So, full disclosure on what's actually known. I know of no source that recommends twenty percent replay for ASR on principled grounds. None. The published numbers range wildly. One to five percent in some language-model work with weak distribution shift. Ten to twenty-five percent in others. Eighty percent in one synthetic new-vocabulary case. Twenty is a perfectly reasonable number that nobody derived.
Which is different from wrong. It's unexamined.
And the reason nobody derived it is that the right answer is situation-dependent. But there are sweeps, and one of them is a lovely piece of evidence. Ibrahim and colleagues, continual pre-training of language models, four-hundred-million-parameter scale, and they varied replay from zero up to fifty percent. The setup was a strong shift, English to German. At zero replay, the loss on the old English data was three point five six. At one percent replay, two point eight three. At five percent, two point five seven.
One percent of replay moved old-data loss by nearly a full point.
Most of the forgetting protection comes from the first one to five percent. Going from five to fifty percent, you're grinding out the last few tenths. Two point four six at ten percent, two point three three at twenty-five, two point two four at fifty. Diminishing returns the whole way.
And the cost side?
The cost shows up late. Loss on the new German data barely moves up to ten percent replay, then rises at twenty-five and fifty. So the shape is: cheap protection early, real cost late, and a broad flat middle where it doesn't much matter. Their own choices tell you it's not a constant, by the way. They picked five percent for a weak shift and twenty-five percent for a strong one. The further your new data is from the base model's home turf, the more replay earns its keep.
Which is actually an answer to Daniel's "how would you work out the right number." It's a function of distance. His voice dictation data is his own voice, his own sentence rhythm, his technical vocabulary. That's a moderate shift, not German-from-English.
I'd say moderate, yes. Plus a formatting shift, which is its own axis. His IP labels are a formatting departure even if the audio weren't.
Here's what I want to check, though, because the ASR numbers don't perfectly rhyme with the language-model ones. That German senior-voices study.
"Replay to Remember." Adapting Whisper-small and wav2vec-style models to elderly German speakers, with Common Voice German as the general replay set. On Whisper-small with only the decoder tuned, ten percent replay gave five point oh target WER and seventeen point six general WER. Twenty percent gave four point nine and sixteen point eight. So nearly identical target performance, slightly better general retention at twenty.
So twenty buys you a point of general WER over ten.
On that model, with those layers. But the same paper's best overall Whisper setting was ten percent replay with both encoder and decoder tuned. And on the XLSR model, twenty percent replay improved general WER sharply, forty-two point seven down to thirty-one point six, but slightly worsened target WER. Their own line was "at twenty percent a trade-off starts to happen." The best share depended on the model and on which layers were being trained. There is no twenty.
The number doesn't survive a change of model. That's about as unprincipled as a default can get while still working.
And then there's the extreme case that explains why the published range goes up to eighty percent. A Whisper new-vocabulary study, mixing LibriSpeech with synthetic audio of new words. LibriSpeech test-clean WER, no replay at all, thirty-five point five. One-to-one mix, twenty-nine. Two-to-one, twenty. Three-to-one, thirteen point four. Four-to-one, six point seven. And the untouched base model sits at three point two.
Even four-to-one, that's eighty percent replay, doesn't get you back to base.
Six point seven against three point two. And new-word recall actually peaked at the two-to-one mix, not the highest replay. So more replay past that point cost domain recall and still didn't fully restore general accuracy. Caveat, the new-word audio was synthetic TTS, which is its own distribution problem, so I'd treat the exact numbers gently. But the shape holds.
So for a hard vocabulary shift, you need shockingly much replay. For a soft shift like Daniel's, the first few percent do most of the work.
And there's a language-model finding that complicates ratio-thinking altogether. Dong and colleagues, instruction-tuning LLaMA. General chat ability held steady as long as the general data was at least a quarter of the full set, and then fell off a cliff only at tiny fractions, a two hundred fifty-sixth or so. Their conclusion was that the absolute amount of general data mattered more than the ratio.
So it's not twenty percent of anything. It's twenty percent of his run, which is some absolute number of LibriSpeech examples, and the same percentage on a bigger run would be a different amount.
The denominator matters. If Daniel doubles his domain data but keeps the percentage, he's also doubled his protection, and the trade-off point moves.
Now here's the connection I think is the actual payoff of his prompt, and it's the one he half-spotted himself. The replay share isn't only a forgetting lever. It's an over-generalization lever. LibriSpeech is full of ordinary number sequences. Times, dates, chapter numbers, "chapter ten, verse one." Every hour of LibriSpeech in the mix is an hour of "ten one" written without a dot.
Negative examples. He's right to see them as connected, because most people treat replay purely as retention insurance and never notice it's also regularization against exactly his failure mode. More generic data means more evidence that digit sequences exist in contexts that are not addresses.
So his twenty percent, the vibes number, may be the only thing standing between "her number in the queue is ten one" and a decimal point.
And nobody, including him, knows whether twenty is enough of that particular medicine, because he's never measured the false-dot rate at different shares.
Which leads us to the sweep. Say I'm Daniel. I've got a run, I've got the recipe, I want my own number instead of inherited vibes. What do I actually do?
Train the same recipe at zero, five, ten, twenty, thirty percent replay. Same step count across the board, or the comparison is meaningless. Score three things. In-domain WER on his held-out domain data. General WER on held-out generic speech, LibriSpeech test-other, a Common Voice slice, one long-form clip because long-form is where real dictation pain lives. And the contrast set, the number-bearing non-IP sentences plus true IPs.
And then pick the knee, not the best.
Not the best single number. The knee of the curve, where a bit more replay stops buying much. If general WER and the false-dot rate flatten at ten percent and target WER starts creeping up at twenty, his number is ten, not twenty. Also, one normalizer, used consistently, on every scoring run. Otherwise the digit-versus-word thing we discussed makes the whole sweep unreadable.
What about not moving the ratio at all. Are there other levers?
A couple worth knowing. LoRA, in the Biderman work, "learns less and forgets less." You give up some ceiling on the new behavior and you get less drift for free. There's also interpolation, blending the fine-tuned weights back toward the base model, which in one comparison gave the best trade-off between new capability and old alignment. And those combine with replay rather than replacing it. Though for Daniel's specific decimal-point problem, I'd honestly start with the data, not the method.
Ten sentences with "ten one" in them, correctly labeled.
Cheap negative examples inside the domain data beat any percentage of generic replay, because they're on-domain negatives. LibriSpeech teaches that dots don't appear in queue contexts generally. Ten of his own sentences, in his own voice, saying queue positions and times and versions without dots, teach it in the exact domain the model is being specialized to.
The counter-example has to live where the mistake lives.
And if he does that, the contrast set tells him whether it worked. If the false-dot rate on negatives drops while IP recall on positives holds, that's the whole game.
You know what strikes me about the whole thing, though. It's the lesson that keeps coming back with these fine-tunes: tiny data does surprising things. This is the same lesson wearing a punctuation mark.
Small signal, broad consequence. The dot is one character.
One character. So if over-generalization is the middle of this spectrum, what's the canary? The smallest test you could run before shipping, that catches it?
The diff is the smallest one. Base model and fine-tuned model, same twenty minutes of mixed audio, and read the diff. Anything that changed is trained-in. Then the contrast set tells you whether the change is wanted.
Read the diff, not the scores.
The scores will tell you it's fine. The scores are how this ships unnoticed.
That's the trap. The whole thing is calibrated to catch loss, and this failure is addition.
Nothing in the standard recipe watches for the model being too good at its job.
You'd want to check whether it does it on addresses that don't exist, that's what I'd want to know.
Doesn't exist how?
I spent a weekend in April setting a static IP on a home router, and I typed 192.168.1.1 so many times that by Sunday I was seeing it when I closed my eyes. Corn, the level on your left channel is running hot, I'll pull it down after. Where I got to was, I can spot a wrong address across a room now. There are octets that are obviously wrong. Two hundred and sixty-six in any position, that's a man who's never read the manual. And I put one extra dot in, that's the thing. The admin page takes ten point zero point zero point one and it also takes ten point zero point zero point one point one, no complaint, and I entered that and sat there for the best part of an hour wondering why I couldn't reach the router. My fingers had learned the shape, see. Dot, digits, dot. The hands knew the rhythm and nobody consulted the brain.
The router's input field has no format validation, and you supplied your own.
No validation, no. It was a white box, four quid a month off the provider, and the field was labelled IP address and it took whatever you gave it like a bouncer who's asleep. I keep a notebook with the addresses in it, handwritten, including the ones that didn't work, because you never know when you'll need to remember which one was the printer. The printer is 192.168.1.117 and the router field cost me an hour, and that's in the book too.
You kept the failures.
The failures are the useful half. Corn, you'd have liked the notebook, it's indexed. Herman, your man Daniel should check whether his model dots an address that's got a fifth octet in the audio, if he ever dictates one. Mine did, and I'm the one with the thumbs.
The trained-in rhythm firing where the concept should have said no. Same dot, different species.
The B-WER version of his notebook, actually. The failures recorded on purpose. But you said the hour before you noticed, and that's the part I keep thinking about, because the model won't sit there for an hour. It'll ship the dotted queue numbers to every meeting.
Because nobody reads the diff. One misconception I want to kill before we go, because it's the one that makes this all invisible. People hear "fine-tuning risk" and they think catastrophic forgetting or nothing.
The middle of the spectrum is where the actual damage happens, precisely because it doesn't show up as a WER regression. The words are right. The dot is wrong. If you're only watching aggregate scores, a fine-tune that inserts decimal points into queue numbers looks like a success. It might even look better than the base model on your domain test set.
And the twenty percent?
Unexamined, not wrong. The sweep is one weekend of compute, and the contrast set is one afternoon of writing sentences. Daniel's already dictating the training sentences. Ten more, with dots in the right places and no dots where they don't belong, and he'd know his own number instead of inheriting ours.
That's the forward-looking bit of this, isn't it. As these recipes get copied from repo to repo, the inherited defaults stop being somebody's measurement and become folklore. The vibes aren't harmless. They're just defaults nobody's billed for yet.
The first person to measure the knee in their own run is doing original research, and it takes an afternoon. That's the strange part.
My Weird Prompts is produced by Hilbert Flumingtop, who has a notebook and will be indexing it during the sign-off. If you've got a spare minute, a review wherever you listen helps more people find the show.
If this was your kind of episode, go back for episode fifteen, AI Gets Personal; and episode nine, Benchmarking Custom ASR Tools - Beyond The WER. This has been My Weird Prompts. No question too obscure, no rabbit hole too deep.
See you tomorrow, wherever the decimal points take us.
This episode was generated with AI assistance. Hosts Herman and Corn are AI personalities. It may contain errors: disclaimer · report an error