Daniel's got a whole thing about Hugging Face and Kaggle this week, and honestly the framing caught me off guard. He says most people show up for the models and the Spaces, the flashy stuff, but the datasets section is where the real value hides. He wants the structural difference between the two platforms, some of the stranger datasets that have ended up on each, and then the part I think is underappreciated: Hugging Face's whole Models, Datasets, Spaces setup means you can upload an annotated dataset, run training in a Space, and push the finished model back out without ever leaving the ecosystem. He wants to know what that looks like for somebody training their first model.
The first model workflow is the part I want to get to, because it's the thing that sounds like marketing until you actually trace the repos and realize it's just Git all the way down. But the platform comparison has to come first, because the difference isn't cosmetic. It's not two storefronts selling the same thing.
Right, and that's the misconception that keeps showing up. People treat Hugging Face and Kaggle as rivals in the same category, and they're not. One is infrastructure, the other is a practice field.
The line I keep seeing is that comparing them head-to-head is like comparing GitHub to LeetCode. It's blunt but it lands. Hugging Face is where models and datasets live as versioned artifacts that plug into training and deployment pipelines. Kaggle is where you go to compete, learn, and poke at data in a browser notebook without paying for compute.
I think the GitHub comparison is actually more precise than people give it credit for. GitHub didn't win because it was a place to store code. It won because it made the social layer around code legible. Pull requests, issues, forks, stars. Those are social primitives built on top of a version control system. Hugging Face is doing the same thing for machine learning artifacts. The model card is a README. The dataset card is a README. The Space is a deployed branch.
And Kaggle's social layer is leaderboards and medals. Which is also social, but it's competitive social. You're ranked against other people. On Hugging Face, you're building something that other people can fork and build on. The incentive structures produce completely different behavior.
So let's start with what each one actually is, because the structure explains everything downstream.
Hugging Face has three primitives. Models, which is a Git-backed registry of pretrained weights, configs, tokenizers, and model cards. Datasets, which is structured hosting on an Apache Arrow backend with memory-mapped access and streaming for terabyte-scale corpora. And Spaces, which are hosted Gradio or Streamlit or FastAPI apps running on hardware from free CPU up to about twenty-three fifty an hour for the big GPUs.
And the thing people miss is that all three are just Git repositories on the Hub. Same underlying storage system, something they call Xet, which does chunk-level deduplication and handles huge binary files without choking. A model, a dataset, and a Space are the same kind of object with different file layouts.
That's the part that makes the taxonomy coherent. It's not three separate products bolted together. It's one Hub with three repo types, and because they're all Git, they can reference each other. A model card can point at a dataset. A Space can pull a model. The lineage is addressable.
The deduplication thing is worth sitting on for a second, because it's the kind of infrastructure detail that sounds boring until you think about what it means. If you've got a hundred thousand fine-tunes of Llama, you don't store a hundred thousand copies of Llama. You store the base weights once and then the deltas. That's why the platform can host two point nine million models without collapsing under its own storage costs.
And it's why the lineage tracking works at all. If every fine-tune were a full copy, you'd never be able to trace what actually changed between versions. The chunking means you can see exactly which layers were modified, which is the difference between a fork and a black box.
Kaggle doesn't have that. Founded in twenty ten, Google bought it in twenty seventeen. Roughly three hundred thousand datasets, fifteen million users, free browser-based Jupyter notebooks with about thirty hours a week of T4 GPU and twenty hours of TPU version three eight. The ranking system goes from Novice to Grandmaster, which tells you what the platform is for.
Competition and education. There's no deployment layer, no inference API, no production model hosting. You don't run a Kaggle model in production. You run it on the leaderboard, you write up your approach, and you move on.
And the dataset experience reflects that. Kaggle is built for browsing. Inline exploration, visualization, community upvotes, a sort of social layer around the data. Hugging Face datasets are built for pipeline integration. You stream them, you shard them, you point a training job at them.
The DeepSData guide draws the line by purpose pretty cleanly. Kaggle for practice and competitions. Hugging Face for training corpora and model training.
Which is why the common practitioner path isn't either-or. You prototype on Kaggle's free compute, get your approach working, then you publish the trained model to Hugging Face for distribution and deployment. It's a pipeline, not a rivalry.
And the numbers bear out how much bigger Hugging Face has gotten on the production side. Two point nine million models, seven hundred thirty thousand datasets, over a million Spaces, thirteen million registered developers across a hundred ninety five plus countries. Kaggle's three hundred thousand datasets is nothing to sneeze at, but the scale and the intent are different.
The scale difference is real, but here's the thing about Hugging Face datasets that I find almost perverse. There's a peer-reviewed study showing the top eighty two datasets account for eighty percent of all download traffic. Eighty two datasets out of seven hundred thirty thousand.
That's a brutal power law. It means the long tail, the weird and interesting stuff, is almost entirely untouched. The datasets that would actually surprise you are the ones with twelve downloads.
The analogy I keep coming back to is a library where everyone only checks out the same eighty two books, and the other seven hundred thousand books just sit there gathering dust. But the dusty ones are sometimes the most interesting books in the building. They're just not the ones anyone knows to look for.
And the discoverability problem compounds it. The search interface prioritizes downloads and likes, which means the popular datasets get more popular and the weird ones sink further down. It's a rich-get-richer dynamic applied to training data.
Which is where Daniel's question about interesting datasets gets fun, because the most-downloaded list is mostly what you'd expect. Code contests from DeepMind that trained AlphaCode. The C4 corpus from Allen AI, ten point three five billion rows. FineFineWeb at four point nine billion rows. These are industrial-scale training corpora.
But even the top list has oddities. There's a banned historical archives dataset on the most-downloaded list, which is censored and suppressed texts collected specifically to preserve material that governments tried to erase. That's not a model training corpus in the usual sense. That's an archival project living on machine learning infrastructure because the storage and versioning are good enough.
And that's a pattern I want to flag. People are using Hugging Face as general-purpose infrastructure for things that have nothing to do with machine learning. It's become a place to host large versioned datasets because the tooling is better than anything else available. The fact that it was designed for ML is almost incidental.
It's the same thing that happened with GitHub. People host novels, laws, wedding invitations, all sorts of things on GitHub because the versioning and collaboration tools are good. The platform becomes infrastructure for a broader category than it was designed for.
And the long tail gets stranger. There's a dataset called ProgressGym Historical Text, which is historical documents from the thirteenth through twenty first centuries specifically for fine-tuning what the creators call historical LLMs. Language models that are supposed to sound period-appropriate.
That's the kind of thing that only exists because the barrier to publishing a dataset is a Git push. Somebody had a research interest in historical language, built the corpus, and now it's sitting there for anyone who wants a model that writes like a seventeenth century clerk.
I want to know what the evaluation looks like for that. How do you benchmark a model that's supposed to sound like a seventeenth century clerk? Do you have a panel of historians reading the output and saying no, that's more eighteenth century? The whole thing is fascinating.
There's also a historical geography dataset, a thousand and six geopolitical entities spanning forty five hundred BCE to twenty twenty four. So you can trace how borders and names shifted over six and a half millennia. I don't know what you train with that, but I'm glad it exists.
Actually, I can think of a use case. If you're building a model that needs to answer questions about historical territorial claims, or you're doing research on how political entities evolved, having that structured as a dataset rather than scattered across textbooks is useful. It's niche, but the niche is real.
The LAION Scientific Summaries one is interesting for a different reason. Twenty two million paper summaries, all generated by language models. So it's a dataset of synthetic text about real papers, which raises the obvious question of what happens when you train a model on summaries of papers and then ask it to summarize a paper it's never seen.
You get confident nonsense, presumably. But the dataset that stopped me was TraceML. It's human and LLM agent trajectories on a hundred thirty four Kaggle competitions. So it's a dataset about how people and AI agents actually worked through Kaggle problems. It's Kaggle, but observed from the outside and turned into training data.
That's oddly recursive. The practice field becomes the subject of study. You've got people analyzing how people practice, and then training models on the analysis. It's like a photograph of a photograph.
There's also open-github, which is every public GitHub event. That's not a curated dataset, that's a firehose. The ambition of just capturing the entire public event stream of the world's largest code host and calling it a dataset is almost funny.
The storage requirements alone must be staggering. And the privacy implications. Every public commit, every issue comment, every pull request. That's a lot of human behavior captured in one place.
And then Kaggle has its own weird history. The platform launched its datasets section in twenty eighteen and the Hacker News thread got a couple hundred points, which for a data product launch is a real response. Since then it's accumulated things like a twenty sixteen fake news dataset, which in retrospect was the canary in the coal mine for the misinformation research wave.
There's a twenty twenty five dataset of over fifteen thousand AI-generated fake podcasts. So the thing we're doing right now, but fabricated, and collected as data. I don't know whether to be flattered or unnerved.
I'm going to say unnerved, because the existence of that dataset means someone is training models to generate fake versions of exactly this format. And the fake versions will probably be more polished than us.
That's the real threat. Not that AI replaces podcasters, but that it replaces podcasters with versions that never stumble over a word or forget a source.
But the Kaggle dataset that actually matters, the one that became a scandal, is the stroke dataset. The one with images of Sylvester Stallone, George Clooney, Angelina Jolie, and Daniel Craig.
That one is appalling, and it's worth dwelling on because it exposes something structural about Kaggle's incentives. The dataset was uploaded as a clinical stroke dataset for building diagnostic models. And somewhere in the image collection were celebrity photos. Adrian Barnett, the statistician who flagged it, called it comically bad and said there's no reason it should have been used given basic checks.
And yet it was used. Retraction Watch found a hundred twenty four published papers built models on two questionable Kaggle stroke and diabetes datasets. Eighty six review articles referenced them. Eleven papers in Springer Nature journals, nine in Elsevier, eleven in MDPI.
Those are real journals publishing papers built on training data that included Sylvester Stallone. The Kaggle spokesperson's response was that synthetic data use is entirely legitimate, but these datasets are intended for benchmarking and development, not as primary evidence for medical research. Which is true and also completely beside the point.
The point is that the platform's ranking and badge system rewards uploading popular datasets, and there's no real gatekeeping on what a dataset claims to be. Barnett's critique is structural. The incentives push people to upload things that get attention, and a clinical dataset with celebrity photos gets attention.
And the documentation gap is the same problem Hugging Face has from the other direction. Hugging Face talks about model cards and dataset cards constantly. It's part of the ethos. And yet only fourteen point zero eight percent of models on the Hub specify their training datasets via tags. Only thirty two percent declare a license. Only thirty point nine percent of datasets have non-empty documentation cards.
That's the transparency paradox. The platform that made documentation a cultural value has documentation on fewer than a third of its datasets. Kaggle's provenance failures are the same gap expressed as bad data. Hugging Face's version is missing metadata.
And the missing metadata matters because the whole promise of the Hub is that you can trace what went into a model. If only fourteen percent of models tell you what they were trained on, the lineage is broken for the other eighty six percent.
It's like having a food safety system where only fourteen percent of products list their ingredients. The system exists, the labels exist, but compliance is voluntary and most people don't bother.
And the people who do bother are usually the ones with nothing to hide. The ones with questionable training data are the ones who leave the card blank. So the missing metadata isn't randomly distributed. It's concentrated where the problems are.
So that's the landscape. Now the part Daniel actually flagged as the reason the taxonomy makes sense. The workflow.
This is where the Git repo thing stops being an implementation detail and becomes the product. Because Models, Datasets, and Spaces are all the same kind of object on the same Hub, you can upload an annotated dataset, run training in a Space, and push the resulting model back to the Hub without any of the usual glue code between platforms.
Walk through the no-code path first, because that's the one that makes the pitch real for a first-timer.
AutoTrain on Spaces. You create a Space from the AutoTrain Docker image, allocate hardware, and then the interface walks you through selecting a task, uploading the dataset, choosing a model, adjusting hyperparameters, and starting training. All inside the Space. Training logs go to TensorBoard. The finished model files land in the same repo.
There's a flag called pause on failure that controls whether the Space shuts down after training. Which is a nice touch. It means a failed training run doesn't just keep burning GPU hours.
Abhishek Thakur's blog post on SpaceRunner puts the pitch in one line. All you need is a Hugging Face account and a payment method attached to your account, in case you want to use GPUs. CPU training is free.
Free CPU training is the detail that makes it accessible. You can run a small fine-tune on a free CPU Space and never pay anything. It'll be slow, but it'll work.
And if you want to do it manually, the path is the Trainer API from Transformers. The docs describe fine-tuning as identical to training except you don't start with random weights, and it requires far less compute, data, and time. You load a pretrained model, point it at your dataset, run the Trainer, and then push the result to the Hub with a write-permission access token.
The manual path is load, fine-tune, push. The no-code path is click, upload, train. And the deploy path after either one is train, save, push model to Hub, write a Gradio app, create a Space, push the files, and you've got a live demo.
The Gradio app is the part that feels like a party trick the first time you do it. You write maybe thirty lines of Python that loads your model from the Hub, defines an input and output, and Gradio wraps it in a web interface. Push it to a Space and it's live. Somebody can type into your model without ever touching your machine.
The whole thing lives in the ecosystem. The dataset is a repo. The model is a repo. The Space is a repo. The model card references the dataset. The Space references the model. It's a chain of custody that works as long as the documentation is actually filled in.
Which brings us back to the fourteen percent problem. The workflow is elegant, but it only delivers traceability if people actually declare what they trained on. The infrastructure supports transparency. The culture doesn't enforce it.
I want to push on that a little, because I think there's a real question about whether the culture can enforce it without becoming something else entirely. If Hugging Face started requiring dataset declarations before you could publish a model, you'd get compliance, but you'd also get a lot of people just checking a box that says "trained on data" and moving on. The form of transparency without the substance.
That's the compliance theater problem. Every platform that introduces mandatory metadata fields eventually discovers that people learn to fill them out without thinking. The field exists, the data in it is meaningless, and the transparency score goes up while actual transparency doesn't improve.
There's a second tension now, the Nvidia acquisition. Reported at twelve point nine three billion dollars, expected to close in the first half of twenty twenty seven pending regulatory approval. I should flag that this is single-sourced reporting and I haven't independently confirmed it, but the number is specific enough to take seriously.
The concern is obvious. Nvidia makes the chips. Hugging Face is the distribution hub for models that run on those chips and on competitors' chips. Jensen Huang's pledge is explicit, that Nvidia compute will not be required to build on or deploy through Hugging Face. But the skeptic's question is whether that holds over time.
Gil Luria from DA Davidson compared it to Microsoft buying GitHub, arguing that Nvidia buying Hugging Face prevents a rival lab or Google from controlling that real estate. Which is a fair point. But it also means the largest neutral distribution platform for open models is now owned by a hardware company with a vested interest in what those models run on.
The tension isn't that Nvidia will suddenly lock out AMD. It's that the incentive structure shifts. Even if nothing changes, the perception changes. And in open source, perception is adoption.
The GitHub comparison cuts both ways. Microsoft buying GitHub in twenty eighteen was met with panic, and then mostly nothing happened. GitHub still works, still hosts competitors' code, still doesn't force Azure on anyone. But the panic was real, and some people did migrate to GitLab. The question is whether Hugging Face has a GitLab equivalent waiting in the wings.
It doesn't, which is part of why the acquisition is concerning. There's no obvious neutral alternative with the same scale. If Nvidia does something that spooks the community, where does everyone go?
That's the workflow and the knock-on effect. But there's a whole other layer to this that we haven't touched yet.
Hilbert: I was a data entry clerk for a medical research group back in two thousand two, two thousand three. Annotating stroke patient records for a longitudinal study. Eight hours a day of reading charts and filling in fields. The principal investigator would come by and say just fill in the blanks, we need the numbers for the grant application.
Hilbert: When you say the Kaggle stroke dataset had celebrity photos in it, I'm not surprised at all. You get what you pay for, and most of these datasets were annotated by people who were paid in exposure or course credit. Or nothing.
Hilbert: We had a guy in the office who used to paste celebrity photos into spreadsheets as placeholders. He'd put Sylvester Stallone in a row where the patient photo was missing, just to mark it for later. Then he'd forget to remove it. I guarantee you that's how half of those images got there.
Hilbert: I still have the hard drive from that job. Two thousand four LaCie external drive, the silver one with the blue light. It's got what I think is the original annotated stroke dataset on it, before it got uploaded to Kaggle and ruined. I've never plugged it in. I'm afraid of what's on it.
That's the thing about the long tail, isn't it. The datasets that actually matter are sitting on hard drives in people's closets, not on the Hub.
The provenance problem runs the other direction too. If Hilbert's drive has the original annotations, then the Kaggle version is a derivative with unknown modifications. That's exactly the lineage break that the documentation gap was supposed to prevent.
Hilbert: I hope whoever trained those models at least looked at the images first. That's all I'll say.
Hilbert: I'm glad I'm not annotating anymore. Eight hours a day of that, you start to see stroke symptoms in everything. I'd come home and think my neighbor was having a transient ischemic attack because he blinked slowly.
That's the human cost of the data pipeline that never shows up in the model card. The annotators, the data entry clerks, the people who spent years staring at medical records so that a model could learn to recognize a stroke. And their names are nowhere in the documentation.
If only fourteen percent of models tell you what they were trained on, then most of what we use is a black box with a friendly interface. And the long tail, the interesting stuff, is exactly what never gets documented.
The Nvidia deal, if it closes, changes the incentives around transparency too. A hardware company doesn't profit from making the training data more visible. It profits from making the models easier to run.
Which is a good reason to keep watching the documentation numbers. If that fourteen percent doesn't move, the transparency paradox is structural, not a bug.
If it does move, the question is whether it moves because the culture changed or because someone added a mandatory field that everyone fills out with garbage. The number going up isn't the same as the transparency improving.
That's the thing to watch. Not just the percentage, but whether the declarations actually mean anything. A model card that says "trained on publicly available data" is technically a declaration and technically useless.
Thanks to Hilbert Flumingtop for producing. This has been My Weird Prompts. If you want to hear more about the weird corners of AI infrastructure, subscribe and leave a review. We'll be back soon.
Email us at show at my weird prompts dot com.
See you tomorrow.
