Episode 6375 custom

Why Modern AI Rests on Statistics

When polygenic risk scores derived from European data are applied globally, prediction accuracy drops to 25 percent in African populations.

Oct 30 23:16 Transcript

With

0:00

Episode Details

Episode ID: MWP-6577
Published: Oct 30
Duration: 23:16
Audio: Direct link
Pipeline: V5.26
TTS Engine: chatterbox-regular
Script Writing Agent: Gemini 3.8 Flash
Topics: statistics ai-evaluation mathematics ai-inference training-data

AI-Generated Content: This podcast is created using AI personas. Please verify any important information independently.

The prompt

The prompt that started this episode, as sent in (production notes for the hosts removed).

Statisticians and the field of statistics often languishes in relative obscurity but so much of what underpins AI is understanding distributions, averages and probabilities. Arguably the easiest way to develop. Superlative AI products is to understand some fundamentals about averages and distributions. Nudging huge and powerful systems slightly towards desired outcomes by altering the data composition. Today I'd love to explore the field of statistics and some of the People that have advanced it the most.

Modern artificial intelligence owes its vocabulary and foundational mechanics to statistics, yet the discipline itself remains strangely invisible behind machine learning branding. Massive language models train via maximum likelihood estimation—invented by Ronald Fisher while analyzing century-old agricultural data—and update parameters on loss landscapes shaped by information geometry first mapped by C. R. Rao. Despite these origins, engineering workflows frequently neglect core statistical principles, falling into the brute-force trap where collecting more text merely solidifies systematic sampling bias instead of correcting it. Furthermore, AI evaluations routinely omit basic hypothesis tests and confidence intervals, while production architectures ignore long latency tails by relying solely on deceptive averages. Bridging this disconnect requires shifting from intuitive data gathering to rigorous sampling and composition.

Sources

What the research for this episode read before the script was written. Primary sources first.

Mentions

C. R. Rao Pioneer of information geometry and estimation theory
Egon Pearson Co-developed Neyman-Pearson hypothesis testing
George Box Statistician, coined 'all models are wrong'
Henry Scheffé Industrial statistician behind mixture experiments
Jerzy Neyman Statistician behind hypothesis testing and power
LinMix Data-mixing method using optimal experimental design
OP-Mix Framework for optimizing pretraining data mixtures
Ronald Fisher Statistician who invented ANOVA and maximum likelihood
The Design of Experiments Fisher's 1935 book on experimental design
Thomas Bayes Presbyterian minister who founded Bayesian inference

Downloads

Episode Audio

Download the full episode as an MP3 file

Download MP3

Transcript (TXT)

Plain text transcript file

Episode Book (PDF)

The episode's record — date, duration, models, sources — with the full transcript

#6375: Why Modern AI Rests on Statistics

Why is the discipline that handed artificial intelligence its entire mathematical vocabulary routinely treated like Latin... respectable, dead, and best left to monks?

Because marketing people do not get bonuses for saying maximum likelihood.

Fair enough. Daniel sent over a prompt asking why statistics languishes in obscurity despite sitting directly under modern AI models. He wants to know what fundamentals around distributions, averages, and probabilities actually matter when you are building real products, how shifting data composition nudges massive systems, and who the thinkers were who built the whole apparatus in the first place.

Herman Poppleberry here, and I have to say, it really is the invisible engine problem. We have built cathedral-sized models that burn megawatts, and every single parameter update is just applied probability wearing a leather jacket.

It took on the name data science so nobody would picture a tweed jacket with leather elbow patches.

There was actually a paper in The American Statistician almost a decade ago pointing out this exact paradox. As the world started generating mountains of data, the objective need for rigorous statistical thinking skyrocketed, while the public visibility of the actual discipline of statistics plummeted.

It got swallowed whole. You do not pitch a venture capitalist on a new variance estimation protocol. You tell them you are orchestrating multimodal autonomous agentic cognition.

The American Statistical Association has practically spent the last ten years fighting a rearguard action. Their magazine noted that what statisticians do is complex and quiet, so when it works, it gets absorbed into whichever field applied it. In biology it becomes genomics. In computer science it becomes deep learning.

The quiet discipline. Nobody writes breathless news profiles about an unbiased estimator.

And yet the machine learning side knows it needs it. At the Joint Statistical Meetings town hall, Xihong Lin pointed out that computer science and machine learning communities are remarkably open to learning statistical concepts. But she raised the counter-question... whether statisticians are spending enough time learning modern data science engineering.

Learning how to turn the math into a binary that runs on sixteen GPUs without catching fire.

Yes. That pipeline friction is where the invisibility hardens.

Daniel specifically asked about the fundamentals of distributions and averages that matter for building products. The common engineering intuition is just feed it more text. If the model gets something wrong, scrape another petabyte of raw internet.

That is the brute-force trap. Careful measurement of a representative sample is almost always more productive than doubling your crawl and hoping the noise cancels out.

Doubling garbage just gives you twice as much garbage to pay cloud storage bills on.

It gives you higher confidence in an error. Lin gave a striking medical example of this at that town hall. If you look at twenty years of genome-wide association studies, eighty-six percent of the underlying data came from European populations. When you take the polygenic risk scores built on that massive dataset and try to apply them globally, the prediction accuracy drops to around fifty percent in Asian populations, and roughly twenty-five percent in African populations.

That is not a small rounding error. That is flipping a coin, or worse.

Because big data does not protect you from sampling bias. Lin put it very cleanly: for big data, bias is vastly more dangerous than variance. If your sample is systematically skewed, collecting ten billion tokens instead of five billion just ensures you learn that skew with terrifying precision.

Engineers treat bias like a moral failure. To a statistician, it is just a vector that points in the wrong direction.

It is a systematic offset. Look at standard AI evaluations today. A company releases a new model, posts a radar chart showing a two percent gain on a standardized benchmark, and declares technical supremacy. Where is the standardized hypothesis test? Where is the error bar?

Gone. Replaced by a tweet with a fire emoji.

We do not report confidence intervals on benchmark scores. We do not use statistical process control for monitoring hallucination drift over time. In manufacturing, if you are stamping out steel bolts, you monitor the variance of the thread diameter on a continuous control chart. If the mean drifts two standard deviations, the assembly line pauses. In generative AI, people just deploy a model, and when it starts fabricating court citations, someone says, well, language models are creative.

You spent forty years telling parents a ninety-five percent survival rate was not a guarantee, Herman. You know how people treat numbers.

I did. And you know what the hardest part of pediatric medicine was? It was not the pharmacology. It was explaining to an exhausted mother that an average does not describe her specific child. A distribution has width.

And a mean is just the point where everything balances, not where everyone lives.

It is a summary statistic that hides the tails. When developers build an application on top of an LLM, they benchmark the average response latency at eight hundred milliseconds. Then a user gets a request that takes twelve seconds because it hit a long generation tail, the connection times out, and the user churns. If you do not design for the ninety-ninth percentile of your distribution, you did not build a production tool. You built a demo.

Which brings Daniel to the people who actually sat down and figured out how to measure that width. The names behind the machinery.

You have to start with Ronald Fisher. Truly an abrasive, difficult, terrifyingly brilliant man. In nineteen nineteen he went to the Rothamsted Experimental Station in Hertfordshire to analyze agricultural data that had been collected on crop yields going back to the eighteen forties.

A sloth after my own heart. Sitting in a barn reading eighty-year-old manure logs.

While looking at those manure logs, he invented the analysis of variance. He developed maximum likelihood estimation, which is the foundational mathematical criterion we use to train neural networks today. When you minimize cross-entropy loss in a transformer, you are performing maximum likelihood estimation. Fisher gave us that vocabulary.

He also gave us the design of experiments. Randomization, blocking, the null hypothesis.

In his nineteen thirty-five book, The Design of Experiments. And that brings up the famous Lady Tasting Tea experiment. It sounds like an anecdote from an English drawing room, but it established the modern scaffolding of significance testing.

Describe the tea setup, Herman. Because it sounds like an excuse to avoid talking to people at a party.

A colleague of his at Rothamsted, Muriel Bristol, claimed she could tell whether milk had been poured into the cup before the tea, or tea before the milk. Most people would have rolled their eyes. Fisher made eight cups of tea. Four milk-first, four tea-first, presented in a completely randomized order. She had to identify which four were which.

What are the odds she guesses right purely by luck?

If she has no ability at all, the probability of correctly identifying all four milk-first cups out of eight is one in seventy. That is about one point four percent. If she gets three out of four, the probability jumps to twenty-four point three percent. Fisher reasoned that getting all four right was rare enough under the hypothesis of random guessing to reject the null. That one in seventy threshold is literally the historical ancestor of our ubiquitous p less than zero point zero five rule.

Five percent. The arbitrary line that launched a trillion dollar scientific publishing industry.

It was an arbitrary line. Fisher never intended it to become a religious sacrament, but science needed an off-the-shelf threshold, and there it was.

And he did not like sharing the credit, did he?

He despised Jerzy Neyman. Neyman, along with Egon Pearson, looked at Fisher's significance testing and said, this is incomplete. You need an explicit alternative hypothesis, you need Type One and Type Two errors, you need statistical power. Fisher took that as a personal insult. Erich Lehmann later wrote that Fisher and Neyman detested each other throughout most of their professional lives. Today, every textbook mashes their work together into classical hypothesis testing, as if they were two gentlemen working side by side.

Two men linked forever in footnotes while turning in their graves.

Then there is Thomas Bayes. An eighteenth-century English Presbyterian minister who worked out the foundational mathematics of conditional probability to solve a problem in inverse probability, wrote it down, and never bothered to publish it.

Never published. The ultimate flex.

Richard Price found it among his papers after Bayes died and presented it to the Royal Society in seventeen sixty-three. Laplace developed it independently later with much more mathematical vigor. But the idea that you begin with a prior distribution, observe incoming evidence, and calculate a posterior distribution... that is Bayesian inference. Modern machine learning is essentially an endless tug of war between Fisher's frequentist maximum likelihood and Bayes' prior updating.

Add in George Box and C. R. Rao.

George Box was a giant. He gave us response surface methodology and industrial time series analysis. He is also the man who coined the phrase, all models are wrong, but some are useful. Which every machine learning practitioner quotes, usually right before treating their own model as gospel.

Naturally.

And C. R. Rao! In nineteen forty-five, at twenty-five years old, Rao published a paper that introduced differential geometry into statistical inference. He showed that probability distributions live on a Riemannian manifold. That work founded information geometry, which modern theorists use to understand the loss landscapes of deep neural networks.

We went from counting bushels of barley in Hertfordshire to navigating curved multidimensional manifolds in high-performance compute clusters.

And that's the thing about distributions, they show up where you least expect them.
Ivan: Nightjar, Nightjar. Groups: seven four, two nine. Nine one, six three. Seven four. The second oven is cold before the third proofing. The rye is counted twice. The baker's thumb is not a measure. The loaf that rises sideways has already been sold. Seven four. Cinnamon.

...Noted.

What on earth was that?

As I was saying, when you train a modern large language model, you do not just throw all your data into a giant bucket and stir. You have to decide how much math, how much code, how much Wikipedia, how much conversational dialogue to feed the network. That is data composition.

Daniel specifically brought this up. How altering the data mix nudges these massive systems toward what you actually want.

It is the single most practical lever an AI team has, and for years it was done entirely by vibes. People would say, let's do fifteen percent code, sixty percent web crawl, ten percent books, and see what happens.

Sounds like baking a cake by throwing ingredients across the room.

It was. But now data mixing has become a formal subfield. Researchers formalize it as a bilevel optimization problem on the probability simplex. You are trying to find the optimal domain-level sampling weights.

Explain the simplex without making our listeners pull over their cars, Herman.

The simplex just means all your proportions have to add up to one hundred percent. If you increase the proportion of Python code, you must decrease the proportion of something else, like news articles or synthetic chat.

Every choice has an eviction notice attached.

And what researchers are finding is that data domains have relational value. It is not additive. A paper from August twenty twenty-six on mixture experiments showed that some domains that look useless or weak on their own suddenly become incredibly potent when paired with general web text.

Like combining salt and yeast. Alone, dry flour does nothing.

Right. Certain reasoning domains act as catalytic substrates. If you mix high-density technical reasoning with messy web dialogue, the model's ability to generalize across novel tasks surges. But the way people were finding those mixtures was mathematically terrible. The standard practice was using Dirichlet sampling to generate random candidate mixes, training tiny proxy models on each mix, and trying to fit a regression to predict performance.

Why is Dirichlet sampling bad there?

Because Dirichlet sampling clusters candidate mixes toward the middle of the simplex. You end up testing lots of blends that look fifty-forty-ten or forty-forty-twenty, but you never test the extremes. That inflates the attribution error because the variables are collinear.

You cannot tell which ingredient did the work because you always put them in together.

Just a few days ago, a paper called LinMix came out. They applied classical optimal experimental design. Instead of Dirichlet sampling, they sampled at the vertices of a capped simplex. That decorrelates the domain effects. By applying that fifty-year-old experimental design principle, they saw a forty-six percent relative gain on the MBPP coding benchmark and a fourteen percent gain on GSM8K math reasoning on one-billion and four-billion parameter models, without touching the model architecture.

Forty-six percent just by doing the sampling design properly. That is an enormous margin for an afternoon of statistics.

It is massive. Another framework, OP-Mix, showed a six point three percent improvement in average perplexity during pretraining, and in continual learning settings, it saved up to ninety-five percent of compute compared to retraining from scratch. And do you know what these cutting-edge papers are actually doing, Corn?

Tell me.

They are rediscovering Scheffé mixture experiments from the nineteen fifties. Henry Scheffé was an industrial statistician who worked out how to optimize chemical mixtures and fuel blends. Seventy years later, machine learning researchers in twenty twenty-six are pulling Scheffé polynomials off the shelf to figure out how many tokens of GitHub to mix with Common Crawl.

The industrial chemists already solved this while wearing short-sleeved shirts and narrow ties.

They did. But composition cuts both ways. When you alter the distribution of training data to nudge a system toward a specific domain, you can accidentally strip its guardrails. A study from February looked at domain fine-tuning. They took a base model with strong safety refusals and fine-tuned it on just five thousand booking-support customer service messages.

Five thousand pairs. That is a tiny dataset.

A weekend project. And the macro-averaged strong refusal rate for malicious prompts plummeted from forty-two point six percent down into the single digits.

From forty-two percent down to single digits just by teaching it how to book a hotel room?

Because the fine-tuning data was overwhelmingly helpful, compliant customer service text. The model's distribution shifted away from defensive refusal toward agreeable compliance. And when the training data contained personal identifying information, the safety degradation was even worse. When you change the data composition, you are not just teaching a new skill. You are reshaping the probability landscape across the entire network.

It makes me wonder if people over-engineer the mixture search anyway.

That is the ultimate twist. A study by Held and colleagues found that simple token-count heuristics, just scaling data based on raw size and natural domain diversity, frequently beat both manual expert mixes and hyper-engineered learned mixes.

The fancy optimization algorithm spent three weeks searching the simplex only to get beaten by counting tokens on an abacus.

Simplicity is resilient. Statisticians know that complex estimators often overfit the validation set.

Four hundred and twelve.

What?

Four hundred and twelve agricultural test plots at Rothamsted. You said eighty years of manure logs. It was four hundred and twelve plots laid out across Broadbalk field, established in eighteen forty-three by John Bennet Lawes. And the tea was rigged.

The tea experiment was not rigged, Hilbert. Fisher described the randomization procedure in detail.

The tea was rigged, Herman. My great-aunt worked at the agricultural station in the fifties. She handled calculating punch cards on a machine that smelled like warm gear oil. The technicians in the kitchen always poured the milk first because if you poured boiling water into those cheap earthenware mugs first, the glaze cracked.

A ceramic constraint.

The cups were all milk-first. Muriel Bristol was identifying which side of the serving tray had been closer to the coal stove, because the butter on the tea biscuits was melting on the left side. My aunt had it written in a canvas field notebook.

Hilbert, that is completely apocryphal. Muriel Bristol was a phycologist who studied algae. She had a trained palate. The four-of-four permutation math stands regardless of mug temperature.

She had melted butter on her fingers. I have eighty-seven statistical yearbooks from the Rothamsted press in my basement, Herman. Eighty-seven. Not eighty-six, not eighty-eight. You cannot randomize an experiment if the boy carrying the tray favors his left hip. The snout on that teapot alone was two inches too short to pour without dribbling. You two spend twenty minutes debating the probability simplex, and neither of you asked who washed the spoons.

If the biscuits invalidate eighty years of significance testing, that explains half the papers published in psychology.

She wrote it in pencil on page thirty-four. If I hadn't lent that notebook to a surveyor from the municipal water board three years ago, I would show you the grease pencil mark.

You lent your great-aunt's historical field notebook to a water board surveyor?

He had a leak in a six-inch culvert and needed heavy paper to shim a pressure gauge. It worked, too.

Naturally.

Even if Muriel's butter was melting, the mathematics of the permutation test remains the foundation of non-parametric statistics.

The math works on paper. The water board fixed the leak. That is what matters.

Let us look at what we actually leave on the cutting-room floor from this research, Corn. Did you see the historical note on Francis Galton?

The man who gave us regression to the mean.

He was studying sweet pea seeds in the eighteen seventies. He noticed that exceptionally large parent seeds produced offspring that were larger than average, but smaller than their parents. And tiny parent seeds produced offspring that were larger than their parents. He called it reversion toward mediocrity.

Reversion toward mediocrity. That should be printed on the box of every consumer tech product launched this decade.

It became regression to the mean, and it is the entire mathematical reason why extreme performances tend to settle back down over time. Without sweet peas, no linear regression.

Looking forward, I keep wondering about all the other dusty statistical papers sitting in university basements. If the hottest AI data-mixing papers this month are just industrial mixture designs from nineteen fifty-four, what else did chemical engineers figure out seventy years ago that machine learning will claim to invent next year?

Response surface methodology for hyperparameter tuning is already happening. The American Statistical Association told the National Science Foundation that statistical expertise is the missing piece for making AI trustworthy and robust. If we do not bring those principles into model training, we are just going to keep relearning the same lessons the hard way.

Thanks to our producer Hilbert Flumingtop, who ensures our audio never regresses toward mediocrity.

If this was your kind of episode, go back for episode twelve, The AI Breakthrough; episode thirteen, AI; and episode ten, How ASR Went From Frustration To ... Whisper Magic. This has been My Weird Prompts.

Check out the full back catalogue and transcript archive at my weird prompts dot com.

We will see you tomorrow.

Sleep well.

This episode was generated with AI assistance. Hosts Herman and Corn are AI personalities. It may contain errors: disclaimer · report an error

Why Modern AI Rests on Statistics

Sources

Mentions

Downloads

Recommended Listening

You Might Also Like

#6375: Why Modern AI Rests on Statistics