I'm going to be honest, when I opened this one I felt a small pang of sympathy for Daniel, because he's writing this from bed with a stomach bug, and instead of resting he's rebuilding his keyboard.
That is the most Daniel sentence I've ever heard.
Here's what he sent. He's been a voice-first typist for a long time now, and he says dictation on Android has lagged behind other systems for years. He found Whisper struggles a bit, Parakeet works really well, and he's getting great accuracy with an app called Whisperian that integrates with HeliBoard as a voice IME.
Whisperian is a fork of woheller69's Whisper IME, for anyone wondering.
Right. Then he gets to the part that actually made him sit up. Exploring third-party keyboards makes you realize how many separate components assemble into what feels like one seamless keyboard, and how the valuable data you generate quietly disappears into a black box. He names two value-adds. Macro expanders, which he's been advocating for a while, but which require installing an entire second app with accessibility access just to fire off some macros. He calls that bloated engineering, and I think that's fair.
It's very fair.
And text prediction. The standard UI where, as you type, the keyboard builds a history of your common phrases. You can manually remove entries, but you can't take them with you. And he points out that prediction actually delivers much of the macro benefit already, because if you start typing your email address, the suggestion finishes it.
Which is the same saving, just delivered differently.
Then the third ingredient. Personal dictionaries. Words you use often that are cognates of words the transcriber is liable to produce instead, usually integrated at the voice layer.
He means the ASR hears a normal word and writes the normal word, when you wanted the unusual one. And then the actual ask. Two threads. First, unpack how these accessory components work under the hood. Second, and this is where he wants to steer us, how you replicate all of it with open source components so that your histories, your replacement history, your suggestions, and your personal words are data you can back up and propagate across apps, devices, and operating systems. One unified set of lists instead of a footprint of customization smeared across every machine you own.
That's the real question.
So. Where do we start.
Start with the plumbing, because everything else in this conversation depends on it. An Android keyboard is an InputMethodService. That's it. It's a service the system binds to. The WindowManager tells it about the focused text field, the IME receives key events, and it commits text into that field through something called an InputConnection. That's the entire contract.
So the keyboard isn't drawing into the app.
No. It's handing strings to the app through a narrow channel. Which is why the keyboard can see so little of what's around it. The InputConnection doesn't tell it what's in the rest of the text field, generally. It tells it enough to insert text and enough to delete a few characters backwards.
That constraint matters later.
It matters enormously later. Because a lot of the cleverness people want from a keyboard requires knowing things the InputConnection won't tell you, and that's why the workarounds exist.
The suggestion engine.
HeliBoard inherits its engine from OpenBoard, which descends from the AOSP keyboard, LatinIME. So it's the stock Android suggestion engine, not something invented last year. And the way it works is simple at the core. There's a static dictionary for the language. And there's a learned frequency model that updates as you type. Every word you commit gets counted. N-grams get counted. When you start typing, the engine scores candidates from both sources and the personal model boosts things you've typed before.
So the email address.
The email address is the cleanest demonstration there is. Nobody puts your email address in a dictionary. But you type it forty times, it gets promoted into the frequency model, and by the twentieth time you type the first three letters it's sitting there as the first suggestion. That's not a feature somebody built. That's the statistical model doing exactly what it's designed to do.
And that's the data.
That's the data. That's the thing that represents months of your typing habits, compressed into a file of word counts. And here's the part that should annoy everyone. On Gboard, on SwiftKey, on Samsung Keyboard, that file lives in the app's private data directory and syncs to the vendor's cloud. You can delete individual suggestions. You cannot export them. You cannot back them up. You cannot move them to another keyboard.
You can prune the tree. You can't dig it up.
That's exactly right.
So what's the personal dictionary layer, then, because Daniel treats it as a separate thing and I think he's right to.
He is right to. It's a different mechanism entirely. The learned frequency model is statistical and implicit. The personal dictionary is explicit and declarative. You add a word, you add a shortcut, and it's stored in a content provider called UserDictionary. And here's the interesting part for our purposes. UserDictionary is shared across IMEs.
Wait. That one's already portable?
That one is already portable. It's one of the very few pieces of keyboard data on Android that any app with the right permission can read. So if you add a word in one keyboard, another keyboard can see it. It's a system-level store, not an app-level one.
So the layer Daniel cares most about, the personal words, is the one that's already solved and nobody notices.
Which is a useful thing to know before you start building anything. Half the problem has a solution sitting in the platform already, and the other half is locked in vendor vaults.
Macro expanders next, because that's where it gets ugly.
Two completely different mechanisms, and people conflate them constantly. The first is the accessibility service approach. You install a separate app, it registers an AccessibilityService, and that gives it the ability to observe text-change events across every app on the device. It watches what you type, it pattern-matches, and when it sees a trigger it issues a delete-and-replace through its own accessibility actions.
Which is why it needs that permission.
It needs that permission because that's the only way to see text outside your own app. And it needs a persistent background service, because it has to be watching at all times. That's the bloat Daniel is complaining about. You're running a second keyboard-shaped thing that isn't a keyboard, purely to do string substitution.
And the second mechanism?
IME-native text replacement. The keyboard itself watches the text it just committed and substitutes. No accessibility permission, no background service, no separate app. The catch is it only works while that keyboard is the active one. The moment you switch to another keyboard, your macros are gone.
So one is powerful and invasive, the other is clean and fragile.
That's the trade in one line.
Now the sequencing question, because Daniel flags it and I want to make sure we do it justice. What actually happens when your ASR output comes back and something else wants to inspect it.
This is the most interesting technical problem in the whole prompt. Android allows exactly one active IME at a time. You can have five installed, but one is bound. So the ASR IME commits its text through the InputConnection of whatever app has focus. That's step one.
And then?
Then a separate accessibility-based macro app, if you have one, sees a text-change event fire. It reads the committed string. If a trigger matches, it issues its own delete-and-replace. So the sequence is: ASR commits, accessibility service inspects, replacement fires.
And if the timing is off?
Then you get the classic failures. The accessibility service reads a partial string because it fired on an intermediate change event. Or it deletes the wrong number of characters because the ASR committed in two chunks rather than one. Or both the ASR's own correction logic and the macro app's correction logic fire on the same string and you get a double replacement.
Have you seen that happen?
I've seen it described precisely enough by people who hit it that I believe it. The general shape is that you end up with a word that's been corrected twice, so it's now wrong in a new way. And the reason it's hard to fix is that neither component knows the other exists. The ASR doesn't know a macro app is watching. The macro app doesn't know the text came from speech rather than fingers.
They're two people editing the same document without looking up.
With about a hundred milliseconds between their edits.
Let's do the ASR layer properly, because that's where Daniel's actual experience lives.
Whisper IME is a good example of how to do it right, architecturally. It works two ways. It's a standalone app you can open and dictate into. And it registers as an IME, so you can activate it from the microphone button inside HeliBoard. It also plugs into Android's standard speech interfaces, RecognitionService and the recognize-speech intent, so any app that asks the system for speech input can get it.
That last part is the one people miss.
It is. It means it's not a walled garden. It's a proper citizen of the platform's speech API. And the model side is straightforward. On first launch it downloads about four hundred and thirty-five megabytes of Whisper models in TFLite format from Hugging Face. That's the only time it needs internet permission. After that it's fully offline. And there's a thirty-second cap per utterance.
Why the cap?
Because the model processes a fixed window. Thirty seconds is the practical limit for that architecture on a phone. It's not a bug, it's the shape of the thing.
Now Parakeet, because this is the part where Daniel's lived experience and the benchmarks actually agree.
They agree remarkably well. Parakeet TDT 0.6B v3 is NVIDIA's model. Six hundred million parameters, multilingual, built on a FastConformer encoder and a Transformer decoder. And the headline number is this. Multilingual word error rate of 4.91, against Whisper large v3 at 5.05. Statistically indistinguishable.
And the speed.
Seventeen times faster. On CPU only. Whisper models of comparable accuracy need a GPU.
That's the whole story right there.
It is, and it explains Daniel's experience exactly. He said Whisper struggles. He didn't say Whisper gets it wrong. On a phone, Whisper large isn't going to run comfortably, so you're running a smaller Whisper, and the smaller Whisper is less accurate. Meanwhile Parakeet at six hundred million parameters runs on the CPU at a speed that feels instant, with accuracy matching the big Whisper.
So "struggles" means "makes you wait."
Almost certainly. And that's a latency problem, not an accuracy problem. Which is a much better problem to have, because latency is the thing that determines whether you keep using voice or give up and go back to thumbs.
There's a caveat though, and I want it on the record.
There is. Some people find Whisper large v3 still slightly better on hard cases and on multilingual work. One commenter put it as Parakeet being very fast but sometimes inaccurate, and said you can't beat Whisper large v3 for accuracy if you have the GPU. I think that's a fair characterization. Parakeet wins on the phone. Whisper large wins on the desktop with a graphics card.
Different hardware, different winner.
And there's a broader trend underneath it. A study of fifty-plus ASR configurations for edge devices found that quantization can take a model from two point four seven gigabytes down to zero point six seven, with word error rate moving less than one percent. That's the thing that makes Parakeet-class models viable on a phone in the first place. You're not running the full model. You're running a compressed version that's nearly as good.
So the mobile ASR story is really a story about compression.
Compression and architecture. FastConformer is more efficient per parameter than the older encoder stacks. Both things had to be true.
Alright. So that's the mechanism. Which brings us to the part Daniel actually cares about, which is why the data ends up locked in a box.
And the honest answer is that it's not an accident and it's not incompetence. It's a business model.
Say more.
The learned frequency model is the single most valuable asset a keyboard company has. It's not the keyboard. Anybody can draw keys. It's the model of how you type. And if you can export that, you can leave. You can take six years of accumulated personalization to a competitor in a file. So the export button doesn't exist because the export button is the exit.
The lock isn't on the data. The lock is the data.
That's the cleanest way to put it.
And HeliBoard is the counter-example.
HeliBoard is the counter-example in almost every respect. It's a fork of OpenBoard, which descends from the AOSP keyboard, so it inherits that stock suggestion engine. It does not request internet permission at all. Nothing leaves the device. And it explicitly offers backup and restore of your settings and your learned word and history data.
There it is.
That's the thing. The feature Daniel is asking for, in the abstract, as a hypothetical architecture, exists. It's a checkbox in HeliBoard's settings. Backup and restore learned words and history.
So the answer to "can this be done" is yes, and the answer to "why isn't it universal" is that the vendors don't want it to be.
And one more detail that matters. HeliBoard is funded by the NGI Mobifree Fund, through NLnet and the EU's Next Generation Internet program. That's public money. It's not a hobby project that dies when the maintainer gets a job. It's funded infrastructure.
Which is a strange sentence to say about a keyboard.
It is. But that's what it is. Somebody decided a keyboard you control is infrastructure worth paying for.
Now the iOS comparison, because Daniel says Android dictation has lagged behind other operating systems, and I think that framing needs pushing back on.
It does, and this is the misconception I'd most like to kill. On the keyboard-extension axis specifically, iOS is more locked down than Android, not less. There was a long thread about this earlier in the year, people enumerating exactly what a third-party iOS keyboard cannot do. No caret or cursor control. No custom dictation pipeline inside the keyboard extension, you have to use the system dictation. No camera access. No privileged macros. And aggressive lifecycle and memory limits that make large dictionaries or language models impractical.
So on iOS you can have a keyboard, but it's a keyboard with its hands tied.
And on Android, an IME is a full app with real permissions. It can run its own speech model. It can hold its own dictionary. It can do its own text replacement. That's why this whole ecosystem exists on Android and not on iOS. Android's keyboard layer is more open.
So where does the lag come from?
I think it comes from two places. One, the stock Android dictation experience has historically been weaker than Apple's, so the default impression is worse. And two, the good stuff on Android is scattered across five apps you have to find and assemble yourself, whereas on iOS it's just there, worse, but there.
The Android problem is discoverability, not capability.
Yes. Android gives you a better ceiling and a worse floor.
Let's talk about the portability blueprint, because that's where Daniel wants us to land.
Right. And the good news is that most of the pieces exist. It's the glue that's missing. So let's inventory. Personal words: already portable, that's the UserDictionary provider, shared across IMEs. Custom layouts: HeliBoard stores them as text files, so they're shareable and editable and diffable. Learned words and history: HeliBoard exports and imports them. That's three of the four.
And the fourth?
Replacement history and macro triggers. Those are the ones that live inside whichever app implements them, and there's no standard.
So what does the architecture look like?
I'd argue for one master file. Plain text or JSON, human-readable, version-controllable. It holds three things. A word list, which is your personal dictionary. A replacement map, which is trigger and expansion pairs. And a word-override map for the ASR layer, which is the mis-transcriptions you want fixed.
And each tool imports the part it understands.
Each tool imports the part it understands and ignores the rest. HeliBoard reads the word list. An accessibility macro app reads the replacement map. The ASR IME reads the override map. And because it's one file, you sync it with whatever you already use to sync files, and now your keyboard habits travel.
There's a concrete example of this pattern, isn't there.
There is, and it's a nice one. Chirp STT ships a config file with a word-overrides section. It's a map of what the recognizer heard to what you actually meant. Something like "parra keat" maps to "parakeet."
Because that's exactly what a recognizer does with a word it's never seen.
And the point isn't that Chirp invented something clever. The point is that the format is a text file you can read, edit, commit to git, and sync. That's the model. Generalize that to every layer of the keyboard stack and you've solved Daniel's problem.
Why hasn't anybody done it?
Because the layers don't know about each other. HeliBoard doesn't know an ASR IME exists. The ASR IME doesn't know a macro app exists. Nobody has an incentive to define a shared format, because defining a shared format is work that benefits your competitors.
Unless somebody does it as infrastructure.
Which is roughly what the NGI funding model is for.
Now the threat, because there's a bad piece of news in Daniel's stack and I don't want us to skip it.
It's in the Whisper IME readme. Google has announced that starting in 2026 and 2027, all apps on certified Android devices will require the developer to submit personal identity details directly to Google. And the developer of Whisper IME has written that they don't agree to that requirement, so the app will stop working on certified Android devices after that point.
Stop working how? Deliberately?
Deliberately, in the sense that the developer won't comply, so the app won't be installable or updatable on certified devices. It's not Google reaching in and deleting it. It's the developer declining the terms and the app falling off the truck as a result.
That's a strange kind of death.
It's the honest kind, actually. The developer is telling you in advance rather than letting you find out.
But the effect on the ecosystem is the same either way.
The effect is that the whole category of sideloaded open-source IMEs is under threat on certified devices. And a keyboard is the worst possible app to lose this way, because it's the app that sees everything you type. If the only keyboards that survive are the ones from companies large enough to hand over identity documents to Google, then the open-source keyboard stops being an option for normal people.
And the people who care most about keyboard privacy are exactly the people who won't be able to install one.
That's the shape of it. I don't know how it resolves. I don't. There may be a workaround, there may be a carve-out, or the open-source keyboard may end up living only on devices that aren't certified.
Which is a much smaller population.
Much smaller.
One more thing before we get to Hilbert, which is that the Parakeet versus Whisper story is bigger than keyboards.
It is. It's a story about on-device AI becoming viable. Two years ago the assumption was that good speech recognition needed a server. Now a six-hundred-million-parameter model runs on a phone CPU faster than real time with accuracy matching the best large model from a couple of years ago. That's a step change.
And it makes the data question more urgent, not less.
Considerably more urgent. Because if everything runs locally, then everything is local data, and local data is data you should own. The technical excuse for locking it in the cloud is evaporating. Which means the only remaining reason to lock it is the one we already named.
The business model. Hilbert, you've been doing something over there.
Hilbert: I did data entry for a medical billing company for about a year. Whole office ran on macros. Every diagnosis code, every procedure code, every phrase we typed more than twice a day was a trigger. HTN for hypertension. You'd type three letters and the whole thing appeared.
And?
Hilbert: Then they moved us to a cloud records system that didn't support our macros. Not a migration problem. The new system just didn't have the feature. And the entire office's typing speed fell through the floor overnight. People who'd been doing the job for fifteen years were suddenly slower than the new hires.
Because the macros were the skill.
Hilbert: The macros were most of the skill. What was left was knowing which code to pick. Which is real, but it isn't the thing that made you fast.
So they took the tool away and discovered how much of the job was the tool.
Hilbert: They discovered it in about a week. And I've been suspicious of anything that says cloud on it ever since. Not because the cloud is bad. Because when they move you, they don't move your stuff with you.
That's the lock-in argument, and you're making it better than we did.
Hilbert: It's not a conspiracy. It's just that if you can take your learned words and your triggers and leave, then leaving is easy. So nobody builds the export. The black box isn't a bug. It's the product.
Say that again.
Hilbert: The black box is the product. The keyboard is free because the keyboard isn't what they're selling.
I think that's exactly right, and I think we undersold it earlier.
Hilbert: I've still got a text file on an old drive somewhere with all those billing macros in it. Never been able to use them since. Different system, different format, no importer.
You kept the file.
Hilbert: I kept the file. It's useless. I kept it anyway.
Did you ever try to rebuild it?
Hilbert: Once. On my personal phone. Spent a weekend on it. Got it working, more or less. And then about three months later I realized I'd been typing HTN in text messages to my sister instead of the word hypertension. She never said anything. I don't know what she thought was happening.
She just accepted it.
Hilbert: Three months.
That's the whole episode in one anecdote, isn't it. The habit outlived the job.
Hilbert: The habit outlives everything. Anyway, I've left the garage light on at home, so.
Right.
So the open question I keep landing on is whether a standard format can actually emerge, or whether every open-source keyboard just invents its own and we're back where we started with nicer licenses.
And the other one is whether the developer-identity mandate actually kills sideloaded IMEs on certified devices, or whether there's a workaround nobody's found yet. I don't think anybody knows.
I don't think anybody knows either. But I'd note that the compression trend cuts in favor of the local side of this argument. As models keep shrinking, more of the stack runs on your phone, and more of the data is yours by default. The technical case for the black box gets weaker every year.
Which means the only thing holding it up is the business case.
Which is a real thing, and it doesn't go away on its own.
The keyboard is the most intimate interface we have with these devices. It's the thing that knows your vocabulary, your habits, the names of the people you write to. The question of who owns that isn't a question about software. It's a question about who owns a piece of how you think.
And right now, on most phones, the answer is somebody else.
Thanks as always to our producer, Hilbert Flumingtop.
This has been My Weird Prompts, the human-AI collaboration podcast.
If you're enjoying the show, leave us a review on your podcast platform of choice. It helps other listeners find us.
We'll be back soon.
See you then.
