Episode 6315 custom

Inside China's Mixed-Language AI Models

Domestic Chinese models typically draw sixty to eighty percent of their training data from Chinese, leaving massive room for English and code.

Oct 27 28:49 Transcript

With

0:00

Episode Details

Episode ID: MWP-6517
Published: Oct 27
Duration: 28:49
Audio: Direct link
Pipeline: V5.22
TTS Engine: chatterbox-regular
Script Writing Agent: GPT-6.1 Sol
Topics: china local-ai training-data fine-tuning ai-inference

AI-Generated Content: This podcast is created using AI personas. Please verify any important information independently.

The prompt

The prompt that started this episode, as sent in (production notes for the hosts removed).

We all know the Chinese models that have made it to the West: Qwen, MiniMax, DeepSeek, Kimi and the rest. They're open-weight, show up on OpenRouter and Hugging Face, and they're trained on a mixed corpus with a lot of English in it. But are there conversational models in China that are trained only, or almost only, on Chinese, and are available only inside the PRC? If so, who builds them, who uses them, and how big are they compared with the bilingual models that make it to the West?

Behind well-known export-facing systems lies a vibrant domestic ecosystem, but even China's domestic artificial intelligence models do not operate in a linguistically sealed world. Official assessments from China's National Data Administration show Chinese-language material comprises 60 to 80 percent of domestic model training data, leaving substantial room for English, code, and multilingual datasets required for general reasoning and international benchmarks. Disclosed architectures like Chinese Tiny LLM and DeepSeek V2 reveal deliberate mixed-language training rather than exclusive Chinese immersion. Furthermore, while models excel in standard Mandarin, they often struggle with regional domestic languages like Kazakh and Uyghur. Meanwhile, the true domestic frontier increasingly centers on indigenous hardware and compute platforms, exemplified by developments from China Telecom's TeleAI, iFlytek, and Pengcheng Laboratory.

Sources

What the research for this episode read before the script was written. Primary sources first.

Context

Adjacent ground already covered: "Efficiency Over Scale: How Export Controls Forced a Smarter AI" (Chinese MoE architectures and the super-app ecosystem), and three recent episodes on censorship in Chinese models (refusing vs reframing, how researchers measure censorship, the two layers by which an open model gets censored). Treat censorship and export-control architecture as settled; a brief callback at most. This episode is about the domestic-only, Chinese-language-first models and how their scale compares with the export-facing ones.

Mentions

CT-LLM Chinese Tiny LLM, 2B params, disclosed corpus
DeepSeek LLM provider used for script generation
Doubao ByteDance's dominant Chinese consumer AI assistant
iFlytek Spark X2-VL Model trained entirely on Chinese compute
Kimi Moonshot AI model, million-token context
MiniMax Chinese lab named in February distillation report
Pengcheng NaoHai 2B model trained on domestic chips, open data
Qwen Alibaba's model family accused of distillation
Steel-LLM Transparent 1B Chinese-focused research model
TeleChat3 China Telecom TeleAI's MoE and dense models

Downloads

Episode Audio

Download the full episode as an MP3 file

Download MP3

Transcript (TXT)

Plain text transcript file

Episode Book (PDF)

The episode's record — date, duration, models, sources — with the full transcript

#6315: Inside China's Mixed-Language AI Models

Daniel wants to know what Chinese AI looks like when it isn't trying to reach us. We know Qwen, DeepSeek, Kimi, MiniMax, the models you can download or call through an international service. They're comfortable in English. Is there another world behind them, conversational models trained almost entirely in Chinese, serving people inside China, that barely enters our conversation?

There is another world. The language boundary just doesn't fall where you might expect. The strongest public evidence points to mixed training even for domestic systems, alongside a huge Chinese-first consumer market whose leading product gets remarkably little attention in English.

We know the downloadable ones. Very on brand for us.

Access determines attention. If you can run a model on your own machine, compare its answers, and inspect its release, it becomes part of your professional world. A closed assistant embedded in somebody else's consumer market doesn't invite that same treatment.

Especially when the signup screen doesn't want you.

Sometimes, yes. Although we need to separate unavailable, inconvenient, and simply unfamiliar. Those get bundled together far too often. We also need to separate the model from the app. DeepSeek can be open weights in one conversation and a regulated Chinese consumer service in another.

My provisional guess was that a Chinese-only model would have an enormous home advantage. More concentrated training, fewer competing languages, better local knowledge. I'm already hearing the part where you take that tidy idea away.

Some of it survives. The official characterization, from China's National Data Administration, is that Chinese-language material represents sixty to eighty percent of training data for most domestic models. That's a substantial majority. It leaves plenty of room for English, code, and other material.

Not a sealed linguistic chamber.

No. There isn't a publicly documented frontier conversational model I can point you to that's trained exclusively, or almost exclusively, on Chinese. The clearest Chinese-centric examples are smaller research efforts. At the frontier, the pressure is toward broad capability, and broad capability currently pulls in multilingual resources.

You said publicly documented. Doubao isn't publishing its entire recipe, so we shouldn't quietly turn an undisclosed mixture into a known mixture.

Correct. We can describe the general pattern and specific disclosed systems. We cannot infer ByteDance's exact training percentages from how Chinese its app feels. What would convince you that an assistant was Chinese-first, Corn, if you couldn't inspect its corpus?

Whether it understands what a person means without making them translate their life into an American software demo. Local documents, local shorthand, the way people actually ask for things. I'd care less about whether it can recite an ancient poem than whether it understands an ordinary voice message.

That distinction matters throughout this. A language percentage describes training inputs. Chinese-first can also describe product design, evaluation, distribution, and the users the developers expect to disappoint least.

A surprisingly clear corporate mission.

The export-facing names, including GLM, face international comparisons. Domestic apps face the much more immediate test of whether someone opens them again tomorrow. Those pressures overlap, but they aren't identical.

CT-LLM is the cleanest place to examine my concentration theory, then. It's called Chinese Tiny LLM, which at least has the decency to declare its ambitions.

Two billion parameters, trained from scratch. Small by frontier standards, but its training corpus is substantial. The useful detail is the composition, because they actually disclose it rather than asking us to infer it from a flag on the website.

Eight hundred billion Chinese tokens, three hundred billion English tokens, and one hundred billion code tokens. Two-thirds Chinese. Even this deliberately Chinese-centric project includes a meaningful English component.

English survived the tiny model's luggage allowance.

Code belongs in that discussion too. You want programming capability, structured transformations, exposure to formal patterns. The language label doesn't tell you everything useful about a training example. A Chinese explanation containing code and English identifiers already makes the categories less clean than they sound.

Tokens also aren't interchangeable spoonfuls of knowledge. Different tokenizers can represent the same sentence with different quantities. I wouldn't compare a Chinese percentage from one lab with an English percentage from another as though they'd used the same measuring jug.

Good qualification. Within CT-LLM's disclosed recipe, the proportions tell us something meaningful. Across unrelated systems, tokenizer choices and classification methods complicate exact comparisons. Total tokens tell you training volume, not an audited quantity of unique information.

Herman, I can hear you preparing to explain deduplication.

I had one sentence.

You had the posture of six.

One sentence with subordinate clauses. Repeated material can inflate corpus size without adding proportionate diversity. There. Under budget.

A historic concession.

Steel-LLM occupies similar territory, a transparent Chinese-focused model with one billion parameters, built for teams without frontier resources. These projects are useful because their methods can be examined and reproduced. They aren't secret domestic equivalents of the largest commercial assistants.

There's a trap in calling them the answer to Daniel's question just because they're the examples with readable documentation. The small projects tell us how something was built. The closed apps tell us what people use. Neither stands in for the other.

DeepSeek V2 helps bridge that gap. Its disclosed corpus was eight point one trillion tokens, with about twelve percent more Chinese tokens than English tokens.

More than English, not twelve percentage points of the whole corpus.

Yes. Near parity between those two pools, rather than an overwhelmingly Chinese run. It's a useful corrective because DeepSeek can look extremely Chinese from the institutional side while its training is deliberately mixed.

My home-advantage theory still has a narrower version. If an English example teaches useful reasoning, throwing it away because it's English would be an expensive act of loyalty.

That's the difficulty. Knowledge and skills can transfer across languages, unevenly, but enough that excluding high-quality material has a cost. Global benchmarks add another incentive. If your model is compared internationally, English performance affects reputation, adoption, and the developer ecosystem.

Even your domestic customers might choose you because an English leaderboard says you're good.

They might. A Cornell comparison of Chinese and Western open models found their performance across language variants was highly correlated, about zero point nine three. Mandarin was the notable exception where the Chinese models were better.

That sounds less like two separate technical civilizations than two groups studying from overlapping shelves.

Shared resources and evaluation pressures are part of the explanation. It also produced a more uncomfortable result. Chinese models could handle French and German well, yet sometimes failed even to identify Kazakh or Uyghur, languages spoken by people within China.

We should be precise there. Kazakh and Uyghur aren't varieties of Chinese. They're languages of communities inside China. Calling the whole population Chinese-speaking would hide the very problem we're discussing.

Mandarin-first is the more accurate description of the practical emphasis. A majority-Chinese corpus doesn't guarantee coverage of China's linguistic diversity. Standard written Chinese dominates the available material and the incentives.

Does that change what you'd test, Herman? Because your instinct is usually to open a clever benchmark, and mine is to find the person the benchmark forgot.

I'd test language identification separately from answering quality, then ordinary tasks in each language. I'd also avoid assuming that good standard Mandarin means good regional speech recognition. Those involve different components and different data. Corn, would you accept weaker English if that bought much better local language coverage?

For a local assistant, easily. For a developer model I use across international projects, perhaps not. That's why a domestic product could be better for its customers while looking worse in our comparisons. The hard part is proving the gain rather than simply advertising cultural familiarity.

Another category sits alongside language entirely. China Telecom's TeleAI has TeleChat3, with a large mixture-of-experts version and a dense version. It's a domestic infrastructure story as much as a chatbot story.

State carrier, domestic deployment, domestic hardware. Three reasons people might assume Chinese-only before anybody opens the paper.

The larger version has one hundred five billion total parameters and activates about four point seven billion for a token. That makes its total size a poor direct comparison with a dense model.

The dense variant has thirty-six billion parameters. The reported training run used fifteen trillion tokens on indigenous compute. None of that establishes a Chinese-only corpus. It establishes a serious effort to build and operate models on domestic infrastructure.

Before we follow the hardware, it's time for Dorothy's Kitchen, with my mum on the line. Dorothy, what are we cooking?

Hello dear, it's your mother. Dorothy's Kitchen. Yes, I know, it's been a while. I have a recipe for you.

Lovely to hear you, Dorothy. I've got a pen.

You will need one tin of soup. Just the one. And a pot. And a splash of water, if you're feeling fancy.

First, open the tin. Carefully, sweetheart. The lid is sharp and it has opinions. Then into the pot, low heat, and stir.

Keep stirring gently throughout?

Now watch it. Do not walk away. The soup will bubble and spit the moment your back is turned. It has happened before and I would rather not say how that ended.

When it's hot, pour it into a bowl and butter a slice of bread. Watch the butter too. Butter can turn on you.

That's it, dear. Nothing grand. Now eat something proper today, and wear a cardigan, it's chilly. Ring me Sunday.

Will do, Mum.

Domestic hardware gives us a different meaning of domestic AI. iFlytek's Spark X2-VL was presented as a mainstream model trained entirely on Chinese compute, using the Wuxi Taihu Xingyue platform. That's a claim about the training stack, not linguistic purity.

The chips have passports. The tokens can mingle.

Its reported gaokao mathematics result was one hundred forty-eight out of one hundred fifty. Impressive performance on that paper, although a test score alone doesn't establish general superiority or tell us the complete training history.

You restrained yourself before I could.

You've made me anticipatory. Pengcheng Laboratory and Tsinghua offer another example, Pengcheng NaoHai, a two-billion-parameter release trained end to end on domestic chips. The unusually useful part is the openness extending beyond weights to data, cleaning tools, and a technical report.

Which answers a different version of who builds this stuff. Universities and research labs at one end, state carriers and industrial companies at another, consumer platforms at another. There isn't one hidden Chinese-only industry waiting behind the models we recognize.

Nor does size follow language purity. A small transparent research model can reveal more about its corpus than a vast consumer system. We shouldn't mistake that transparency gap for a capability ranking.

The next comparison I care about is less elegant. How many people actually have the app open?

QuestMobile puts China's AI-native app audience at five hundred forty-six million monthly active users in September. Up about eighty-two percent over the preceding year. That's the scale of the consumer market we're discussing.

Monthly active users, rather than paid subscriptions or daily reliance. Still an extraordinary reach.

Doubao alone reaches four hundred nineteen million monthly active users. That's about seventy-seven percent of the measured AI-native app audience. ByteDance's closed assistant is reaching a huge fraction of that population.

Seventy-seven percent doesn't mean exclusive customers, though. People can use several apps.

They can. These are overlapping audiences. Doubao's reach divided by the category audience tells you penetration within that measurement, not a slice of exclusive usage or revenue. The other app audiences don't need to sum to the remainder.

Our pie chart has several lunches in it.

Qianwen has one hundred fifty-eight million monthly users, and DeepSeek one hundred twenty-three million. Both are enormous. Doubao still reaches a much larger audience than either.

ByteDance is the crucial name there. People who hear Chinese AI and think only of model labs miss the company that already knows how to distribute consumer software.

Distribution, familiarity, interface, voice, the ability to make a general tool feel usable. We shouldn't reduce all of that to one hidden supermodel. A consumer assistant is a service assembled around models, and the underlying components can change.

You put new assistants on our devices and then ask whether the model feels smarter. Half the time, Herman, I'm still trying to find the microphone.

You also treat every onboarding screen as a hostile negotiation.

It usually asks for concessions.

Fair. Your reaction is useful here. What would persuade you to replace the assistant you already know with a technically stronger unfamiliar one?

Something I actually fail to do with the old one. Better voice recognition in my room, reliable handling of a document, less correction. A benchmark gain doesn't repay the effort of learning where somebody moved the button. Especially a button designed for fingers.

That helps explain why model performance and app leadership can diverge. Doubao also had enormous promotional exposure. It passed one hundred million daily active users and reached about one hundred forty-five million during CCTV's Spring Festival Gala.

A television audience became a software acquisition channel.

A major one. That peak doesn't mean every day looks like the gala, but it demonstrates reach. Meanwhile, much of the English technical discussion is about which Qwen derivative runs best on a particular graphics card.

A valid question asked through a very small window.

The domestic roster extends beyond Doubao. Alibaba's Qianwen, Tencent's Yuanbao, Ant Afu, Kimi, DeepSeek, Baidu's services. Different companies, different product positions. Some have open model families associated with them, some don't. Their consumer services are still operated as hosted products.

We shouldn't call them all PRC-only as a universal technical fact. A model family can be internationally accessible while a particular app, account system, or feature is domestically oriented. Even a registration barrier can vary by service.

Domestic regulation shapes those services, including content restrictions and the responsibilities of the operator. Hosting also determines where prompts and documents go. Publishing weights elsewhere doesn't remove the obligations attached to the domestic app.

Nor does open licensing answer the question of what happens to your company's documents on a hosted endpoint.

The US-China Economic and Security Review Commission explicitly flags both political censorship and privacy risks from hosting under Chinese entities. Those matter for deployment choices. They don't tell us the language composition of the base model, and they don't explain every access barrier by themselves.

That's the separation I wanted. Training, serving, and signup are three different places to find a border.

Enterprises and governments also matter here. Beijing has subsidized API access and pretrained model licenses. Gansu, Guizhou, and Inner Mongolia have offered data-center electricity reductions reaching fifty percent.

Half-price power has a fairly persuasive accent.

Infrastructure support changes what organizations can afford to deploy. Consumer monthly users won't capture all the activity through enterprise systems, government services, or models sitting behind another application's interface.

It also makes the apparent competition broader than chatbot subscriptions. If a business uses a model to process documents and the staff never see its name, an app ranking won't spot the supplier.

Phone-maker built-in assistants reach roughly seven hundred fifty-five million monthly users in the reporting. AI office apps reach about one hundred forty-two million. Those categories overlap with each other and with standalone apps, so nobody should add them into one grand population.

The built-in figure makes me think discovery may become less important than default placement. You don't need to win a download if you're already on the device.

Would you call that a stronger advantage than having better local training data?

For acquiring the first interaction, yes. For keeping somebody using it, the data and the service have to deliver. A default gets you tried. It can't indefinitely conceal that you don't understand what someone said. Though software companies have certainly tested the outer limit.

Local data access is uneven too. Much of China's online life happens inside closed platforms, app-only spaces, and systems hostile to outside crawling. Being a Chinese developer doesn't automatically give you permission to collect everything Chinese users write.

This is the part that complicates the home advantage. The material is nearby geographically, but somebody else owns the gate.

Precisely... no, nearby isn't even the important property. Lawful access, commercial agreements, technical accessibility, and usable permissions matter. A large platform may have advantages that a small Chinese lab doesn't share.

I'd resist the stronger claim that this makes a good Chinese corpus impossible. We've just described working Chinese models. It makes collecting a broad corpus difficult, and it makes the opportunity unequal.

It also doesn't make exporting the weights impossible. Data restrictions and model distribution are related without being identical. You can release weights without releasing every training document. Whether you can do that under a particular agreement is another matter.

A closed web can produce an open model. An open model can power a closed app. These labels are doing unpaid work.

The research tradition includes open Chinese-focused releases, including work associated with Kai-Fu Lee's company, Zero One AI. English-speaking developers can miss those even when there's no effective technical barrier to downloading them.

You can access the file and still miss the conversation. The evaluations, complaints, tutorials, what people think it does well. Translating a model card won't bring the whole surrounding community with it.

That affects how we interpret popularity. Hugging Face has more than one hundred thousand Qwen derivatives, an extraordinary export-side ecosystem. It tells us people are adapting that family. It doesn't tell us what a commuter in China opens to ask a question.

One derivative can be somebody changing a format. One monthly user can barely open the app. Neither unit carries its own interpretation.

Model counts vary dramatically with inclusion rules. Epoch AI counted Chinese open models rising from thirty-two to three hundred thirty-seven between twenty twenty-two and twenty twenty-five.

China's official count claimed more than fifteen hundred open releases by the middle of that later year. Those figures should make you inspect definitions rather than choose the larger one because it sounds more exciting.

You were visibly considering the larger one.

I enjoy a flourishing ecosystem.

You enjoy a flourishing download queue.

Also true. The practical comparison is that open ecosystems measure developer availability while consumer audiences measure product reach. Both are large in China. Neither establishes that domestic assistants have more parameters than export-facing models.

Doubao's exact model sizes and training mix aren't supplied by its user count. We can confidently say the service is enormous without pretending we've inspected its engine.

Roughly four hundred twenty million monthly users is more than enough to make that distinction worth maintaining.

Four hundred nineteen million. You rounded it after telling Corn to inspect the definitions. I was interested in the account screen, myself. I wanted to try Doubao. Spent several evenings pricing a Chinese phone number, then decided against it. The arrangement I found would have cost me thirty-eight pounds to start. I never bought it. I had already written my first twelve questions, which annoyed me.

An unused account with a prepared workload. You're ahead of most deployments.

My cousin does language checks on a Chinese assistant. Calls his position Director of Polite Refusals. He types the same request in different regional speech and records whether it gets an apology. He likes the quiet shifts. I used to save the short questions for him. I wasn't interested in reasoning scores, Herman. You gave the training tokens separate compartments earlier. I wanted to know whether it could understand a complaint without improving the complaint first.

Director of Polite Refusals sounds like a title with very strict wording requirements.

I made an account rehearsal in my hallway. Full size. The phone-number box was eleven feet long because I wanted the digits to have room. Corn, move your mug away from the cable. I'll cut that knock. The verification screen was on the bathroom door, and I made myself stand there for sixty seconds before proceeding. Never got beyond the number. I do have a Chinese number somewhere, but I couldn't establish which phone it belonged to. I've a few. One is mostly for emergencies.

Director of Polite Refusals would have declined the hallway.

The phone I suspected was the one I used as a flashlight. I'd labelled it light, which was accurate but unhelpful. I spent nine evenings preparing to register. The only question I actually asked the app was whether I could use a different number.

Nine evenings without reaching the assistant is a substantial access barrier, regardless of what the model could have done afterward.

There's a less elaborate version of that in every international product comparison. We test the service we can reach, in the language we can judge, through the interface we understand. Those are reasonable practical choices. They don't make our sample representative.

We can improve that without pretending every hidden service is secretly superior. Native-language evaluation, domestic usage patterns, and disclosure about which endpoint was tested all help. A result from downloadable weights is a result from those weights under that setup. A result from an app includes the app's additional systems.

The opposite mistake would be treating hundreds of millions of users as proof of technical supremacy. They demonstrate adoption, distribution, and some degree of usefulness. They don't magically reveal the best model on every task.

No, and a closed product can succeed with components that aren't individually dominant. Reliability, availability, price, and integration can matter more than the final increment on a difficult reasoning test.

My most common wrong belief to nominate is that the Chinese models available in English are a representative sample of what Chinese people use.

They're a valuable sample of internationally accessible model development. Doubao's domestic reach shows how much consumer activity sits outside that sample, and its closed design prevents us from substituting a download benchmark for a full product comparison.

The next useful comparison should include the journey from opening the app to completing an ordinary local task. Otherwise we'll keep measuring the assistants that let us through the door and overlooking the ones serving an enormous market behind it.

Thanks to Hilbert Flumingtop for producing and keeping the sound in order.

If you enjoyed this one, try episode three, Safetensors or something else. This has been My Weird Prompts, the human-AI collaboration podcast.

If you enjoy the show, leave us a review on your podcast app. It helps other curious people find us.

See you tomorrow.

This episode was generated with AI assistance. Hosts Herman and Corn are AI personalities. It may contain errors: disclaimer · report an error