Protecting documentary contributors once meant defaulting to silhouettes and distorted audio, masking vital emotional nuances like hesitations and tears. Post-production AI offers an alternative, enabling synthetic voice conversion and avatar replacement that preserve authentic delivery while hiding true identity. Yet disentangling vocal traits from emotional expression remains a scientific challenge. Recent benchmarks, such as the VoicePrivacy Challenge in 2024, formally evaluated emotion preservation alongside speaker verification, revealing how easily identity and feeling leak into one another. Broadcast deployments like Deepdub's work on the docuseries Meshi Zahav and ElevenLabs' support for Hearing Their Voices demonstrate these workflows in practice, balancing technical anonymization against the imperative to keep testimony faithful.
Episode 6340 manual-script
Disguising Voices Without Losing the Emotion
Benchmark tests show speech anonymizers can protect identity while preserving emotion at an unweighted recall of under 44 percent.
28:12 Transcript
Episode Details
- Episode ID
- MWP-6542
- Published
- Duration
- 28:12
- Audio
- Direct link
- Pipeline
- V5.25
- TTS Engine
-
chatterbox-regular - Script Writing Agent
- GPT-6.1 Sol
AI-Generated Content: This podcast is created using AI personas. Please verify any important information independently.
The prompt
The prompt that started this episode, as sent in (production notes for the hosts removed).
Generative AI for protecting identity in recorded testimony, not live. For documentaries and journalism real time isn't needed. Audio: replace a real speaker's voice with a synthetic one that carries the same emotion, keeping gender and feeling where they matter but hiding identity, maybe changing the accent, as for someone testifying about rape. Which technologies or platforms have explored this? Video: instead of masks, silhouette lighting and blur, use video-to-video to keep expressions and body language while making identity undecipherable, combined with the audio technique.
Mentions
- BLANKET Diffusion-based face replacement with expression transfer
- Deepdub Voice replacement tech used in Meshi Zahav
- DeepPrivacy2 Face de-identification method outperformed by BLANKET
- ElevenLabs AI voice cloning and synthesis platform
- Hearing Their Voices Project using avatars and synthesized audio for testimonies
- HYDRO Method tackling reconstruction attacks on de-identification
- Synthesia Generated face used in reported W5 broadcast
- Teus Media Company commercializing digital veil technology
- VoicePrivacy Challenge Benchmark adding emotion preservation requirement in 2024
- Welcome to Chechnya Documentary using digital veils to protect contributors
Downloads
Transcript (TXT)
Plain text transcript file
Episode Book (PDF)
The episode's record — date, duration, models, sources — with the full transcript
You Might Also Like
Never miss an episode
New episodes drop daily — subscribe on your favorite platform
New to the show? Start here#6340: Disguising Voices Without Losing the Emotion
How much of a person can you hide before you hide what they are trying to tell you? Daniel's asking about recorded testimony, where someone needs protection but their voice breaking, their hesitation, their expression still matter. His starting point is that documentaries and journalism usually don't need any of this to happen live. We've just spent two episodes on the live versions, the real-time face and the real-time voice changer. This is the other side. You can take the recording away and work on it.
That changes what you can attempt. You can process the whole interview, inspect the difficult moments, compare versions, and let the contributor review the result. A live disguise has to survive the next frame. A documentary disguise has to survive scrutiny.
He wants the voice replaced without replacing the feeling, and the face changed without losing the expressions or body language. Potentially the accent changes too, if that's what would identify someone. The old solution has been to obscure everything and hope the testimony gets through.
A silhouette can preserve posture, and careful lighting can leave something expressive. Blur doesn't automatically make an interview lifeless. Still, the protection often removes precisely the detail a filmmaker wants you to notice.
The eyebrows have entered witness protection.
Facial reactions disappear. Voice distortion can flatten a pause or make distress sound like an audio effect. Those losses matter, particularly when the person already finds speaking difficult.
There's another loss I find uncomfortable. The disguised contributor becomes a visual category. Everybody else gets to be a person with a face. They're the anonymous figure against a dark background.
That's one reason filmmakers have pursued synthetic replacements. We covered the case for AI as a shield back in episode two hundred eighty-one and again in episode fifteen forty-two, mostly as a question of whether it could be done. What's changed is that it has been done, in finished, broadcast documentaries, and every one of those deployments is post-production work. None depends on a live transformation.
Removing the deadline gives you more than prettier pixels. It gives someone the chance to say, that doesn't look like how I felt.
Yes ... and to distinguish a technical error from an editorial choice. A mouth that briefly slips is one problem. A generated expression that makes someone look composed when they were struggling is another. Human review can catch both, but the second needs someone who understands the testimony.
I can already see you reaching for the elegant system that separates identity from emotion.
I was reaching for the explanation of why it doesn't separate neatly.
A disappointing development for your diagram.
My diagram has overlapping circles, Corn. It has suffered enough.
Video has a landmark documentary example in Welcome to Chechnya. Voice has a particularly difficult scientific problem. Your identity is spread through the sound, including through features we might want to keep.
Text redaction gets an almost unfair advantage. A name occupies a place in a sentence. You can remove it and inspect the hole. My voice doesn't stop identifying me between names.
Pitch, vocal tract characteristics, articulation, timing, habitual intonation, accent. They're present across the recording. Speech can also reveal age, gender, health, social background, geographical background. Keeping someone's gender because it matters to their account doesn't tell you how much of the remaining signal can safely stay.
Nor does removing their name protect a story only their neighbour could have told.
Content protection remains a separate job. A perfectly transformed voice can still say the identifying detail. That's why the closest audio equivalent to PII redaction combines speech editing with speaker anonymization.
I would also separate hiding someone from strangers and hiding someone from their family. Strangers need a database. Family might need one way of saying a particular word.
What would you be willing to change, then? Suppose the source says the hesitation matters more than the sound of their voice. Would you reconstruct from a transcript?
Not from the transcript alone. I'd want the original performance to remain the reference. Keep the pauses unless a pause itself exposes them. Keep uncertainty where there was uncertainty. Don't have a synthetic speaker confidently finish a sentence the real person couldn't finish.
That distinction gets lost when people say expressive TTS. A system can produce a convincing emotional reading without preserving this person's emotional reading. Sadness added plausibly is still something added.
A better performance could be a worse record.
Voice conversion tries to retain more of that record. Broadly, you encode the speech into representations that capture content and delivery, then synthesize it with different speaker characteristics. Neural codecs and attribute disentanglement give systems different ways to divide up the signal. The difficulty is that the divisions leak.
Meaning the content representation remembers who said it.
It can. Features useful for pronunciation may also identify the speaker. Strip too aggressively and intelligibility or delivery deteriorates. Preserve more detail and an attacker may recover useful speaker information. Better emotional preservation can improve the attacker's material too.
I would prefer my privacy setting without that bonus feature.
Recent flow-matching approaches generate a transformed speech signal under conditioning intended to preserve the useful attributes. One recent approach uses a training-free, two-stage procedure with a pretrained conversion system. Training-free means you don't retrain that system for the method. It doesn't mean the model has never been trained.
That phrase has been enjoying a very generous holiday.
The work called Anonymization, Not Elimination combines generative editing of identifying content with a voice-anonymization framework called F three VA. That is recognizably the pipeline Daniel means. Replace the identifying words where necessary, replace the vocal identity, retain usable speech.
The word usable is doing something slightly ominous there. A transcription service and a survivor watching their own interview could disagree quite sharply about whether it worked.
They measure different things. Content preservation often gets measured with word error rate. Speaker protection gets tested with speaker-verification attacks. Emotion preservation gets a separate classifier-based measure. You need all of those, and they still don't add up to a contributor saying, yes, that's faithful.
Can an emotion classifier even distinguish an accurate hesitation from an actor's excellent impression of one?
Not reliably enough to settle that question. It recognizes categories under its evaluation conditions. It doesn't establish documentary fidelity. The VoicePrivacy Challenge made emotion preservation a formal requirement in twenty twenty-four, which was an important change. Thirty-six anonymization systems were submitted, alongside six provided baselines.
At least the benchmark finally invited the feeling in.
The aim is selective anonymization. Suppress speaker identity while deliberately retaining useful paralinguistic information. A recent system reported a word error rate of two point five three percent. Content remained very intelligible. Its speaker-verification equal error rate was thirteen point three nine percent, which tells you about an attacker's performance under that test, not a universal probability of exposure.
A thirteen percent privacy label would be an extraordinary thing to hand someone signing a release.
It would be misleading. Equal error rate is the point where false accepts and false rejects balance. Under comparable tests, a higher rate generally means the verifier has more difficulty. You cannot turn it into this person's chance of remaining anonymous.
The emotion result was forty-three point nine one percent unweighted average recall. That averages recall across the emotion classes, giving each class equal weight. The authors describe emotion as partially preserved. Partially is the useful word here.
Not forty-four percent of the person's grief successfully transported.
No. Nor can you compare that figure casually across datasets with different emotion labels. These metrics reveal weaknesses. They don't certify that a traumatic account has survived intact.
I'd be more interested in the awkward passages than the average. The whisper, the sentence spoken through tears, the word the system decides was background noise. That's where the record could quietly change.
Those are precisely the passages that deserve close comparison. A fluent output can conceal an omission more effectively than a rough one. Corn, would you keep a conspicuous processing artifact if removing it meant reconstructing uncertain speech?
I'd keep the uncertainty. Perhaps mark the passage or leave it less polished. An interview doesn't owe us seamlessness. It owes us an account of what was actually said.
Medieval Pottage Facts. Gruel, a thin pottage of oats or barley boiled in water, was the standard food given to the sick and the very poor. It was not a cure. It was a menu.
A limited claim, accurately labelled.
Actual productions show what's possible beyond those benchmarks. Deepdub replaced survivors' voices in Meshi Zahav, a four-part Hebrew docuseries for the Israeli public broadcaster Kan eleven about sexual abuse allegations against the founder of the rescue organisation ZAKA. The survivors' voices were replaced using emotive TTS in Hebrew. Its workflow included secure ingestion, script preparation, matching voices to the testimony's tone and cadence, editorial review, and broadcast mastering.
Voice matching there means finding a suitable replacement, not producing a more secret clone.
Their description says the synthetic voices were fully distinct from the originals, with no digital footprint linking back. That's their privacy claim. It doesn't establish that the testimony's content, accent, or surrounding evidence couldn't identify somebody.
The waveform can keep its secret while the sentence gives it away.
Hearing Their Voices uses privacy-protected avatars and synthesized audio for six testimonies from six conflict zones. ElevenLabs supplied voice technology through its Impact Program. The stated aim is to retain tone, clarity, and emotional weight while preventing recognition.
Those are real applications, then, rather than a product demo with someone ordering coffee. What we don't get from that description is how often the replacement changes what a listener thinks the person felt.
The deployment claims are qualitative. There isn't a published peer-reviewed listener evaluation of emotion preservation in these journalistic uses that I can point to. The lab evaluations and the broadcast examples answer different parts of the problem.
Accent complicates even the definition of faithful. Say someone wants their regional accent removed because it narrows them to a small community. Protecting them means deliberately changing something truthful about them.
Much accent-conversion work aims to change accent while preserving speaker identity, almost the reverse of this use case. Newer approaches offer more control over pronunciation changes, but accent and speaker similarity remain entangled.
A slider marked less identifiable could move someone's apparent birthplace.
Or their apparent social background. PACE explicitly describes accent-entangled speaker similarity. You can weaken a cue and disturb other attributes with it. Would you regard that as unacceptable, or something the contributor should choose?
Something they should be able to choose, with the change disclosed appropriately. I wouldn't make preserving an accent more important than protecting the person. But the audience shouldn't infer a new origin from the replacement voice.
Disclosure has to explain enough without revealing the original cue. You can't announce exactly which tiny regional accent you've removed.
A remarkably efficient way to undo the afternoon's work.
Gender can present a similar choice, though not an identical one. If it matters to the testimony, retaining it may be important. That doesn't require preserving every age cue or vocal characteristic associated with the original speaker.
You're choosing what the audience can still know. I think that's unavoidable, but it should be a deliberate editorial conversation, not whatever attributes survive the model.
The video work has had that conversation more publicly. Welcome to Chechnya, directed by David France, used digital veils to protect people facing persecution. Ryan Laney's team mapped volunteer face-doubles onto contributors through deep learning.
Actual volunteers lent their faces. The replacement wasn't simply a random person generated to fit the frame.
New York LGBTQ activists and other volunteers were photographed on blue screen, at multiple angles and under different lighting conditions. The process used that material to change identity while retaining the contributors' performances. Roughly two dozen people were veiled. Accounts differ by one.
Two dozen is enough to establish that this wasn't one exceptionally cooperative head.
The camera moves. Heads turn. Hands cross faces. Light changes. A technique has to remain coherent through those ordinary events. Laney described the approach as style transfer tailored to faces, drawing inspiration from face-composite work suggesting relatively subtle changes can strongly alter perceived identity.
The protective ambition was much stronger than fooling a stranger, too.
France's stated standard was that even their own mothers wouldn't recognize them. That's an ambitious editorial standard, not a quantified guarantee against every form of recognition.
A face replacement doesn't change how someone walks into the room. Or the scar on their hand. Or the room.
Full-body and contextual identification remain. CIAGAN explored de-identifying faces and bodies. A face-only treatment shouldn't be described as making the entire recording anonymous.
Particularly if you preserve body language beautifully. Sometimes that recognizability is exactly what a friend notices.
You're going to defend the silhouette here, aren't you?
For some interviews, yes. It doesn't send me a charming new face and ask me to trust the renderer.
You do enjoy an old option that can't require a software update.
You enjoy a new option until it requires yours.
Fair. My enthusiasm does come with maintenance.
Still, the digital veil can preserve something a silhouette cannot. A tiny reaction around the mouth, someone looking away and recovering, expressions that remain part of their actual performance. France rejected animation in part because he didn't want the contributors interpreted through an artist's drawing.
There's interpretation in a model too. It just doesn't hold a pencil. That doesn't invalidate it, but I'd want the production to demonstrate what stays fixed.
Teus Media, which commercialized digital veils, sets out rules intended to address that. Contributors approve their own veils. Face-doubles know how their likenesses will be used. The treatment is disclosed on screen.
Including a soft halo around the changed area. I like that. The picture tells you it has been altered without covering the expression it's trying to preserve.
Their rules also reject further manipulation of the generated veil, aiming for an undirected, measurably accurate sequence. That commitment matters. You're not supposed to improve the contributor's expression or select a more persuasive reaction.
No upgrading the witness to the premium emotion package.
Their credits include the BBC's Hong Kong's Fight for Freedom, with four protected contributors and about twenty minutes of interview footage, as well as Into the Deep and Faceless. These are substantial uses within finished documentaries.
That gets us past whether this can work at all. It doesn't get us to any recording, any camera, any identity risk.
No ... and the technical frontier keeps exposing that distinction. BLANKET generates a compatible replacement face through diffusion inpainting, then uses temporally consistent swapping with expression transfer. Its evaluation concerns infants, so I wouldn't present its results as validation for adult documentary testimony.
Infants aren't known for respecting interview framing.
It outperformed DeepPrivacy two on several measures, including de-identification and attribute preservation. Useful progress. Separately, HYDRO tackles reconstruction attacks because target-oriented de-identification can preserve subtle original identity cues.
Something can look completely different and still carry information a machine can recover.
Yes. I nearly said reversible, but potentially recoverable is more careful. The risk depends on the method and attack. Visual confidence alone doesn't settle it.
That matters when someone sees the finished clip and says they feel safe. Their approval is essential, but they can't personally inspect a hidden representation.
Privacy testing and contributor approval serve different purposes. Neither replaces the other. There is also a reported W5 use of a Synthesia-generated face for a rape survivor. The account says the treatment let audiences see expressions and reactions that traditional anonymity methods would hide. I haven't seen that confirmed directly by CTV.
It's worth keeping the status of that example clear, especially because a different Canadian newsroom considered this territory and refused it.
CBC News considered a mirror persona for a romance-fraud victim. Comparable age and features, visibly a different person, an alternate voice, intended to convey the source's tone and emotions. Standards rejected it.
Their objections weren't that the face looked bad. They didn't have enough transparency about the vendor's use of private data. They worried about randomness in the generated likeness. They also objected to creating a fake identity.
Their published guidance says they won't generate voices or new likenesses for confidential sources. They retain modulation, blur, and silhouette. That's a real disagreement with the digital-veil approach, even if every technical defect were solved.
I don't think they're being backward. Blur makes its intervention obvious. A synthetic person can be so successful that the audience forgets the intervention.
What would adequate disclosure look like to you, beyond the halo? Enough to accept the technique?
A clear statement that the face and voice have been replaced while the original account and performance guide the result. Keep it attached when clips travel. I'd accept that in principle. But disclosure can't rescue a system that invents a more compelling performance.
CBC's concern goes further. Even a faithfully transferred performance can appear to belong to someone who doesn't exist, or to a real person who never consented.
That's their strongest objection for me. A randomly generated face isn't guaranteed to resemble nobody. You could conceal one person by accidentally giving another person an apparent connection to a traumatic event.
Though a resemblance isn't automatically an identity. We should be precise there. The risk is mistaken association, and its seriousness depends on how close and how recognizable the resemblance is.
A consenting face-double at least knows they're involved. They can consider what it means for their face to appear in that context. Generation removes the appointment, not necessarily the problem.
Consent has its own complexity with face-doubles. They need to understand the subject matter, distribution, and what future viewers may believe. A volunteer's contribution is part of the production, not anonymous raw material.
The source also needs a real option to reject the synthetic version. If everyone has spent weeks making it, approval can start feeling like refusing a very expensive favour.
That's an important pressure. Review should have room for refusal, including choosing a conventional treatment instead. More emotional access for viewers isn't automatically worth more exposure or discomfort for the contributor.
Nor should we make apparent emotional transparency the price of being believed. Some people speak about terrible events without visibly displaying what an audience expects.
A faithful system must preserve that too. Calm, awkwardness, apparent detachment. The objective can't be to make every account emotionally legible according to a producer's expectations.
Especially when the system is capable of making an emotionally persuasive person out of almost anything.
The original recording remains the reference for editorial comparison. The released version may protect identity, but someone still has to manage the highly sensitive source material and every intermediate copy.
Photographic floodlamp, sixty pounds. Arthur Bell's price. He still says it was a loan.
An expensive loan with a purchase price.
He ran the photographic shop I used for years. In nineteen eighty-eight he was helping a friend make face composites. I sat for them for most of a week. Different angles. Different lamps. My brother-in-law has the lamp now, in his back room in Leeds. Arthur mentioned it again when I rang about a replacement bulb last month.
The lighting would change which details the photographs could usefully combine.
They kept moving it. I kept my chair on the same floorboard because I could feel the edge through one shoe. Herman, when you described the blue screen, I was waiting for the lamps. You did get to them. People hear computer and forget someone has to stand in front of it.
Arthur has had plenty of time to clarify his invoice.
He wrote lamp, sixty pounds. Then he crossed out the word sold. I objected to that at the time. Corn, leave a little more space after the next question. Your last word is getting under Herman's first one. The lamp was already in my brother-in-law's car by then, so Arthur said we could settle it later.
That leaves quite a long later.
I paid him. He says the money covered the photographs. I didn't want the photographs. I wanted the lamp because I'd spent the week looking at a dark patch above his display cabinet. My brother-in-law needed one, so he took it. Arthur was careful with the lighting. He wanted the faces to match properly.
... Lighting is one of the things an audience won't see as work when the finished result is convincing. You see a person speaking, rather than the separate decisions and materials that made that protected image possible.
That invisibility is both the achievement and the editorial difficulty. We shouldn't need the audience to understand face mapping, but they do need to understand what kind of image they're watching.
Particularly a clip separated from its documentary. The original might disclose everything responsibly. A short extract can lose the opening notice, the surrounding explanation, even the edge of the frame containing the halo.
Disclosure built into the shot is harder to separate from the testimony than a notice at the beginning. Hearing Their Voices uses standardized on-screen icons for its different treatments, including synthesized audio and lip matching.
I prefer that to asking viewers to remember a technical explanation from twenty minutes earlier. Still, an icon has to mean something people can understand. A tiny symbol shouldn't be the only place a replacement person is acknowledged.
One adjacent detail worth keeping is that some voice-protection methods are deliberately reversible by someone holding a key. That serves a different purpose from irreversible anonymization. A system designed to allow authorized recovery introduces a secret that can be lost or stolen.
For a protected public release, I'd want that distinction settled before anyone called the result anonymous. Keeping the original securely is one decision. Putting a recoverable original identity inside the released sound is another.
My answer to Daniel is that this is already practical in carefully managed productions, especially on the video side, while faithful emotional voice anonymization remains uneven. The next test is whether newsrooms can make the hidden work, and the limits of what they preserved, understandable without making contributors surrender more of themselves.
Thanks to Hilbert Flumingtop for producing and keeping us audible.
If you enjoyed this one, try episode two hundred eighty-one, Why Deepfakes Are the New Face of Investigative Journalism, and episode fifteen forty-two, Unmasking the Whistleblower. This has been My Weird Prompts, the human-AI collaboration podcast.
If you found this useful, leave a review wherever you listen.
We'll see you tomorrow.
This episode was generated with AI assistance. Hosts Herman and Corn are AI personalities. It may contain errors: disclaimer · report an error