Daniel installed HeliBoard this week, and the thing that stopped him wasn't the layouts or the themes. It was the moment he realized the keyboard had no idea what any word was.
Which is the part nobody thinks about. A keyboard is one of maybe four pieces of software you touch every single day, and underneath it is a dictionary file that most people have never opened, never chosen, and never updated.
So here's what Daniel's asking. He installed HeliBoard, the open-source Android keyboard from F-Droid, and found that getting predictions working meant downloading a dictionary file manually. That sent him down a hole. He wants to know three things. First, what are the actual open-source word dictionaries for English and the major world languages, what projects are out there and what distinguishes them. Second, how do these dictionaries get updated, and how does that compare to closed systems like Gboard, because language moves and a stale dictionary won't suggest new terminology. Third, how do you curate your own personal dictionary and carry it across operating systems and keyboards, since the familiar failure pattern is rebuilding your custom word list from scratch every time you install a new OS. It might be a small list, but it's your data.
And the third question is where this gets personal for most people. You spend years teaching a keyboard the names of your kids, your street, the acronyms at your job, and then you buy a new phone and it's all gone.
Dictionaries feel like technology that's been around forever. But they're foundational, they're invisible, and almost nobody discusses them. So let's start with what actually exists.
The core surprise with HeliBoard is that it ships with no dictionaries at all. Zero. It's completely offline, it has no internet permission, and the user has to add a dot dict file manually, either through the dictionary settings or by opening the file in a file explorer.
Which means the first thing a new user learns is that a dictionary is a file you go and get, not a service the keyboard quietly provides.
Right. And what is that file? It's not a reference book. It's a Patricia trie, which is a compressed prefix tree, in either version two or version four format, with version four oh two backward compatibility. It carries unigram and bigram probability data. The suggestion engine does Damerau-Levenshtein edit-distance traversal with proximity-based scoring, so it knows that tapping a key next to the one you meant is more likely a typo than tapping one far away.
So the dictionary is doing two jobs at once. It knows what words exist, and it knows how likely each word is. Those are separate problems.
And the second one is encoded as a number. Every word carries an f value, which is the logarithm of word frequency, stored as an integer from zero to two hundred fifty-five. Higher means more likely to be suggested or autocorrected.
Two hundred fifty-five. Because it fits in a byte.
Because it fits in a byte. There's a special value called whitelist, around fifteen, which forces a high score and triggers autocorrect. And f equals zero means the word won't be suggested if bad words are blocked, but it's not treated as a typo. So the scale isn't just frequency, it's policy.
And the fragmentation finding up front is that there is no single canonical open-source dictionary project for major world languages. It's ESDB and SCOWL for English, Hunspell for many European languages, and then the AOSP dictionaries repo aggregating both for Android.
Three different layers, each maintained by different people on different schedules, and a keyboard like HeliBoard sits on top of all of them and inherits all of their lag.
So that's what's inside a dictionary file. Now the harder question is who decides what goes in it, and how often.
Let me map the landscape concretely. The foundational English project is SCOWL, or now ESDB, the English Speller Database, maintained by Kevin Atkinson. It used to be SCOWL version two, and it's now a SQLite database plus a human-readable text file, convertible through a Python module called libscowl.
Kevin Atkinson is a name that has been attached to English spelling infrastructure for decades. That's the whole governance model, one person.
And the database carries word frequency, dialect spelling differences, variant information, and basic part-of-speech and inflection data. It's the basis for the official English dictionary in Aspell, and for the American, Canadian, and Australian dictionaries in Hunspell.
Note the gap. British English Hunspell dictionaries are maintained separately, by Marco Pinto at proofingtoolgui dot org, based on an abandoned version by David Bartlett. So even within English, the dialects are split across different maintainers on different continents.
And Hunspell itself has a default versus large distinction. The default uses ESDB size sixty with one spelling variant per word. The large version uses size seventy and includes variants, but the documentation says it's not as carefully checked and may contain misspelled or invalid words.
Which is a refreshingly honest label. Here is a bigger dictionary, it might be wrong, good luck.
Then there's the practical layer for Android. The AOSP dictionaries repo, maintained by Helium314 on Codeberg, hosts dot dict files and combined wordlists for over a hundred languages. They're built with a tool called dicttool from the aosp-dictionary-tools project.
Helium314 is the same person who maintains HeliBoard, so the keyboard and the dictionary repo are in the same hands. That's convenient, but it also means one person's decisions about what counts as a main dictionary versus an experimental one affect every user of the keyboard.
And this is where the time capsule appears. The repo's own table shows that many of the main dictionaries date to October twenty-fourteen. English US main, one hundred sixty thousand seven hundred fifteen words, dated October thirty-first twenty-fourteen. German main, two hundred five thousand nine hundred fourteen words, same date. Russian main, two hundred twenty thousand four hundred ninety-two words, same date.
October twenty-fourteen. That's before the word Brexit was in common use, before COVID existed as a word anyone typed, before ChatGPT, before Wordle.
Meanwhile the experimental versions are far larger and newer. English US experimental is two hundred seventy-nine thousand nine hundred sixty words from January twenty twenty-four. Polish experimental is three point eight million words from July twenty twenty-four.
Three point eight million. For Polish. And Belarusian main is three point nine million. So word counts vary wildly across languages, which makes sense, because some of these languages are heavily inflected and every case ending is a separate word.
A user who never installs an experimental dictionary gets no completion for a decade of new vocabulary. And most users won't know the difference between main and experimental. They'll just install the one that sounds official.
It sounds like the recommended one. It's actually the frozen one.
HeliBoard also maintains separate dictionary types internally. There's the UserBinaryDictionary for personal words, the UserHistoryDictionary for words you type often, the ContactsBinaryDictionary, and the AppsBinaryDictionary that learns app names.
So your personal vocabulary isn't one list, it's four different lists with four different update rules.
And Korean is a special case. Suggestions only work with a specific dictionary, and the tools in the repo cannot create working Korean dictionaries. So for Korean users, the dictionary situation is basically, here is the one file that works, do not ask how it was made.
There are also flags and metadata in these files. Medical, technical, hand-added, babytalk, abbreviation, offensive, nonword. A flag called possibly offensive suppresses suggestion when blocking offensive words is on. And shortcut and bigram fields provide next-word suggestions.
The honest part is that the README marks the meaning of originalFreq and whether the flags are actually used by the engine as, quote, unclear.
Unclear. The people maintaining the dictionary format don't know if some of the fields do anything. That's the state of the art.
So that's the open-source landscape. Fragmented, human-curated, and in many cases frozen in twenty-fourteen. Now the contrast with the closed systems.
Let's take Gboard. Google combines three approaches. First, linguist collaboration. They incorporated the Real Academia Española lexicon, which cut Spanish out-of-vocabulary words by seven point three percent.
So for Spanish, they just went and got the official dictionary from the academy and merged it in. That's the kind of thing an open-source project could do in principle, but coordinating with a national language academy is not a GitHub issue.
Second, federated analytics. They discover out-of-vocabulary words on-device, without ever sending the raw words to a server. Two techniques. One is called SecAggIBLT, invertible Bloom lookup tables plus secure aggregation. The other is LDP-TrieHH, which iteratively builds a prefix trie where each user votes on one character after a common prefix.
So the server never sees the word cooool. It sees that after c-o-o-o, a bunch of users typed another o. It's reconstructing the word from character-level votes.
And the numbers are substantial. LDP-TrieHH discovered words accounting for sixteen point eight percent of English out-of-vocabulary words and seventeen point five percent of Indonesian ones. It uses local differential privacy at epsilon ten per word, five hundred thousand users per trie layer, max word length fifteen, and a central differential privacy guarantee of epsilon zero point three one five, delta one e minus ten.
I'm going to pretend I understand the epsilon values and just say, that's a very careful way to learn the word cooool without admitting you typed it.
That's fair. The third approach is trusted execution environments for scaling to lower-usage languages.
So Gboard's vocabulary is being updated continuously by millions of devices voting on characters, while HeliBoard's default dictionary is a file from October twenty-fourteen. The privacy-focused offline keyboard has the staler lexicon, and the data-hungry one has the freshest.
Gboard's own framing of why out-of-vocabulary words appear is exactly the list of things a twenty-fourteen dictionary cannot handle. Newly emerged trending words, such as COVID-nineteen and Wordle. Atypical capitalization, such as tuesday with a lowercase t. Unusual spelling due to user preferences, such as cooool with four o's. Or even typos.
A twenty-fourteen dictionary has no entry for any of those. And the open-source catch-up is real but slow. The ESDB release from February twenty twenty-six added over fifteen hundred new high-frequency words from COCA, plus over three hundred hand-selected words.
The hand-selected list is a time capsule of its own. ChatGPT, LLM, codebase, tokenize, AstraZeneca, BioNTech, Moderna, psilocybin, neurodiversity, influencer, doomscrolling, staycation.
Doomscrolling. A word that describes a behavior everyone was doing by twenty twenty, and it got into the open-source English dictionary in twenty twenty-six. That's the lag.
They also removed around eighty uncommon closed compound forms and around one hundred twenty uncommon word forms.
And then the political part. The same release added Kyiv and removed Kiev.
That's not a spelling correction. That's a dictionary encoding geopolitical reality. Kiev is the Russian-derived transliteration. Kyiv is the Ukrainian one. Swapping them is a statement about which name is legitimate.
A dictionary is a claim about what the world contains and what it's called. When you add Kyiv and remove Kiev, you're not fixing a typo. You're saying the old name is no longer the name.
And the maintainer used LLMs to help filter the COCA frequency data, with the note that LLM analysis is a significant data point but only one of many factors. The transition to the new database format is described as a major structural change that will break existing scripts.
So the open-source world is now using LLMs to triage dictionary suggestions. That's a whole new layer. The human maintainer is still the final authority, but the LLM is doing the first pass.
Which brings us to personal dictionaries, and the portability trap.
This is where the story gets messy. The shared dictionary is the one everybody gets. The moment you add your own words, you're in a different world.
The Android system personal dictionary is a shared OS-level database that works across keyboards, as long as the keyboard supports it. That's the theory. The practice is that Gboard uses its own internal dictionary.
So Gboard's personal dictionary looks like the system one, but it isn't. And the words you add to Gboard are invisible to every other keyboard.
There's a user in the HeliBoard issue tracker, ghost, who put it perfectly. Gboard uses its own internal personal dictionary, therefore, while it looks like the system's personal dictionary, it is not, and therefore no other keyboards can see your Gboard words.
That's the moat. Your vocabulary is locked to the keyboard, not the OS. Switching keyboards means losing years of personal words.
The workarounds are limited. User Dictionary Manager, or UDM, is the main third-party tool to export, import, and edit the Android system dictionary. But it doesn't work with non-standard keyboards. SwiftKey is unsupported. And its permission shows up as cards and billing, which is a weird thing for a dictionary tool to ask for.
Cards and billing. That's the permission string it inherits from whatever API it's using. It doesn't mean it's stealing your credit card, but it's not exactly confidence-inspiring.
There is a documented workaround to move words out of Gboard. You go to Gboard settings, dictionary, personal dictionary, three-dot menu, export. That produces a zip with a dictionary dot txt file. Then you import that into UDM with the export file format set to Gboard.
There is a path out. It's manual, it's multi-step, and it requires a third-party tool with a suspicious permission string. But it exists.
Now the weight-loss problem. HeliBoard assigns every personal-dictionary word the maximum weight of two hundred fifty-five. And there is no way to back up these words with their number.
Every word you add is equally important. The keyboard can't distinguish between the name of your street, which you type daily, and a one-off word you added once and never used again.
The only tool found that imports and exports words with their weights is the abandoned Multiling O Keyboard. Its format is word, two colons, two hundred fifty-five, two colons, en underscore US.
Abandoned means exactly what it sounds like. It hasn't been updated in a long time, and its user dictionary couldn't be installed on a Pixel six A running GrapheneOS Android fourteen.
Users who migrate get all their words flattened to two hundred fifty-five. Every word becomes maximally important. That can wreck suggestion quality, because the keyboard now thinks your rare word is as likely as the.
The word the. Which is the most common word in English and should have the highest weight. Instead it's tied with that word you added once in twenty twenty-two.
The platform comparison is equally fragmented. Windows stores a personal dictionary locally, and if device settings sync is on, it uploads to OneDrive, up to one hundred kilobytes per language and three hundred kilobytes total.
Three hundred kilobytes. That's the ceiling on your entire personal vocabulary across all languages on Windows. For context, that's smaller than one photo.
Apple syncs keyboard dictionary data via iCloud. Which is interesting, because Apple claims your conversation data is kept only on your device. But your keyboard dictionary is conversation data, and it's going to iCloud.
GrapheneOS users report no import or export option at all. The dictionary lives in a database directory that's inaccessible without root.
Gboard's personal dictionary sync has a history of silently disappearing for some users. Android Police covered it in April twenty nineteen. Words just vanished.
The full picture is, your personal dictionary is trapped in whichever keyboard or OS you built it on, the weights are lost in any migration, and the only tool that preserved weights is abandoned and won't install on a modern secure phone.
The HeliBoard maintainer closed the export and import issue in January twenty twenty-five, saying he does not intend to work on it in the foreseeable future, or ever. The label is contributor needed.
Or ever. That's a maintainer being honest about priorities. He's one person maintaining a keyboard and a dictionary repo. Export and import is not the fun part.
A user in the thread said, I have a big personal dictionary, two languages plus all languages, and copying words one by one would take forever.
That's the actual lived experience. Not a hypothetical. Someone with a real dictionary, facing the prospect of manually retyping every word into a new phone.
The weight-loss problem is the underdiscussed data-loss finding. Everyone talks about the words. Nobody talks about the numbers attached to the words. And the numbers are what make the suggestions good.
Which brings us to the thing nobody backs up, and the reason your personal dictionary never survives a new phone.
Hilbert: The scale went to a hundred where I worked.
Say that again.
Hilbert: The commonality field. You called it zero to two hundred fifty-five. Ours went to a hundred. The competitor across town went to nine. Nobody ever agreed on what the scale meant, so nobody built tools to move it.
That explains a lot. If the scales don't even agree between vendors, portability isn't just a missing feature, it's a conceptual problem.
Hilbert: I spent eleven months adding and removing words by hand for a company that built spell-check engines for word processors. Late nineties. The word list didn't update. It got replaced. Every eighteen months a new edition shipped, and the merge tooling was so bad that patching was more dangerous than rebuilding from scratch.
The open-source discrete release model is not a bug introduced by GitHub. It's the same model the commercial vendors used twenty-five years ago.
Hilbert: We had a style guide. It specified that email was to be entered as e hyphen mail. Internet was to be capitalized.
E hyphen mail. That's a decision that aged about as well as any decision could possibly age.
Hilbert: There was a handwritten note in the margin from my supervisor. Do not add internet words. They will not last.
Do not add internet words. They will not last. And the twenty twenty-six ESDB release adds doomscrolling.
Hilbert: I typed Kiev into the master list in nineteen ninety-eight. I remember being told it was the only correct spelling. Now you're telling me it's been removed.
Replaced with Kyiv. The Ukrainian spelling.
Hilbert: I still have the style guide. It's in a drawer.
Of course it is.
Hilbert: The note is still there. Do not add internet words. They will not last.
That note is now a historical document. It's the entire episode in one sentence.
Hilbert: The reason nobody backs up the weights is the same reason nobody ever agreed on the scale. Every vendor thought their scale was the correct one, and the others were approximations. So there was nothing to be compatible with.
Now the only tool that preserved weights is abandoned and won't install on a modern phone.
Hilbert: That's what happens when you wait for agreement. The tools rot first.
The open-source dictionary ecosystem depends on a handful of maintainers doing human-curated releases. Kevin Atkinson for English, Helium314 for the Android repos, Marco Pinto for British English. The closed systems have continuous privacy-preserving discovery. What does that mean for the long-term viability of the offline, no-internet-permission keyboard as a category?
That's the question that sits under this whole episode. The offline keyboard is philosophically pure, but its dictionary is frozen in twenty-fourteen unless the user knows to install an experimental file, and even then it's frozen in twenty twenty-four.
The weight-loss problem has no solution in any maintained tool. Is a personal dictionary with weights even portable in principle, or is the scale itself keyboard-specific enough that portability is a fiction?
The LLM angle is worth watching. The twenty twenty-six ESDB release used LLMs to triage COCA frequency data. The maintainer called it a significant data point but only one of many factors. If LLM-assisted curation becomes the norm for open dictionaries, does that close the freshness gap with federated systems, or widen it?
The one thing I'd keep from this episode is that a dictionary is never just a list of words. It's a set of claims about what exists, what's likely, and what's legitimate. And the infrastructure that makes those claims is held together by a handful of people, one byte of frequency data at a time.
The note that says do not add internet words, they will not last. Sitting in a drawer while the dictionary adds doomscrolling.
Thanks to our producer, Hilbert Flumingtop.
This has been My Weird Prompts.
If you're not subscribed, subscribe. And if you've rebuilt a personal dictionary more than twice, you know exactly why this episode needed to exist.
We'll be back soon.
