Daniel's got a whole thing this week about who actually gets paid to test hardware, and it starts from a place I remember well. He says years ago he did freelance tech writing, and one of his niches was reviewing VPNs. It left him with what he calls a jaundiced view of the market and an unused virtual machine on his laptop. His point was that the review process was real — he actually tried the VPNs, made notes on features, kept detailed logs on latency and speed across endpoints — but from the outside, it looked identical to something just sponsored by the vendor. And he says the whole thing felt like a waste of time, because the checklist-driven grind was indistinguishable from the affiliate marketing swamp. His question is basically threefold. Who gets paid to test hardware, and how does someone land that job? Is real testing becoming a niche luxury for the few outlets that can afford it? And given that AI has gotten good at surfacing real community sentiment and skipping the affiliate garbage, is there room for a renaissance of true human reviewers?
I want to start with the VPN part, because I think Daniel's experience there is the cleanest window into the whole problem. He did the work. Logs, latency, speed tests, feature checklists. And he's right that the output was indistinguishable from sponsored content. But here's the thing — the reason it was indistinguishable is that the sponsored content was copying the same checklist format. The affiliate sites learned to mimic the structure of genuine testing without doing any of it. So the genuine work gets buried under a thousand thin imitations, and the reader can't tell the difference without spending an hour cross-referencing.
That's the part that would make me quit. Not the tedium — the futility.
Right. And Daniel's asking whether that's still the landscape. So let me give you the honest answer. It's worse and better at the same time. The economic incentives haven't changed, but the tools for cutting through the noise have. But before we get to the AI part, let's actually answer the first question. Who gets paid to test hardware?
Because that's the thing people actually want to know. It sounds like a dream job. Someone hands you a drill, you use it, you write down what happened, you get paid.
And the answer is that almost nobody gets hired directly to do that. The people who test hardware for a living fall into a few distinct buckets. You've got the professional editorial outlets — Wirecutter, Consumer Reports — and they employ staff testers. Consumer Reports has been doing this since nineteen thirty-six, and it's subscriber funded, no advertising. Wirecutter has full-time staff who spend weeks on a single product category. Those jobs are real, they pay salaries, and they almost never hire someone off the street. They hire journalists, engineers, people with domain expertise.
So the job posting for drill tester doesn't exist.
It barely exists. What exists is a job posting for a staff writer who can also test drills, and they expect you to come from a trade publication or an engineering background. Then you've got the YouTube reviewers — Project Farm is the obvious one. That channel tests tools with measurable outcomes. He's got a rig that applies consistent torque, he measures things, he shows you the data. And he's viewer funded. Ad revenue, channel memberships, no manufacturer sponsorships. That's the model everyone points to as the clean one.
And he built that audience over years by being trustworthy. Which is the hard part to replicate.
Then there's Amazon Vine. That's the program where Amazon gives free products to reviewers they've identified as trusted, in exchange for honest reviews. No cash, just product. And it's been criticized for years because the incentive structure is weird. You get free stuff for writing positive-leaning reviews, and even if you're honest, the selection of who gets what is opaque. And then there's the B2B testing world — trade publications that test drills for contractors, and those testers are often paid by manufacturers or industry bodies. So the whole landscape is a spectrum from independent to fully captured.
So when Daniel asks how to get the job, the honest answer is you don't apply for it. You build a reputation doing something adjacent, and the testing becomes part of a larger role.
And the freelance testing gigs he did — the VPN work — that's the murky middle. Those gigs exist because affiliate marketing created a demand for content that looks like testing. The company doesn't care whether you actually tested the VPN. They care that you can produce a review that ranks for the right keywords and converts. Daniel's experience of actually testing and keeping logs was him doing more than the job required, and the job didn't reward the extra work.
Which is why it felt like a waste of time. He was producing genuine work for a system designed to reward the appearance of work.
And that's the core economic problem. Real testing is expensive. You need time, equipment, expertise, and you need to buy or source the products. Affiliate revenue is nearly free by comparison. You write a review based on the spec sheet, you stuff it with keywords, you slap on an affiliate link, and the page earns money whether or not the review is accurate. The margins on thin content are enormous. So over the past decade, a huge number of outlets that used to do real testing shifted to the thin model, because the market didn't punish them for it. The reader couldn't tell the difference, so why spend the money?
The tragedy of the commons, but the commons is consumer trust.
And that's where Daniel's observation about Epicurious comes in. He mentioned that in a past episode we talked about Epicurious doing field testing for kitchen gear and came away impressed. That's the model that still works. They had real cooks using the equipment in real kitchens, not just checking specs. And that kind of testing is expensive and slow. It's exactly the niche luxury Daniel suspected it was becoming.
So the question becomes whether AI changes the calculus. Daniel's point was that AI recommendations seem to skip the affiliate garbage and surface real community sentiment. Is that actually true?
It's partially true, and the reason is interesting. AI models trained on vast amounts of web data end up with a kind of statistical averaging. If a thousand affiliate sites all say the same thing about a VPN, but Reddit threads and forum posts from actual users say something different, the model's training process doesn't simply average them out. It learns to weight certain patterns as more informative. Community language — people complaining about specific bugs, mentioning workarounds, comparing real-world latency — has a different texture than marketing copy. And the model picks up on that texture.
So the AI is doing what Daniel did manually. Reading through the noise and finding the signal.
In a sense, yes. But there's a paradox. The Product.ai trust report from last year found that around seventy percent of consumers now say they trust AI-generated recommendations more than human reviews. That's a striking number. But the same report highlighted that AI can be gamed too. The training data includes the same affiliate garbage, and there's already a cottage industry of people trying to manipulate what AI models say about products by flooding the web with favorable content.
So the AI's edge is aggregation, not infallibility.
That's the right way to put it. The AI is better at averaging across thousands of sources, but it's not immune to a coordinated campaign to poison the well. And it lacks something else. Hands-on nuance. An AI can tell you that a drill has a twelve volt motor and a half inch chuck. It cannot tell you that the chuck loosens after twenty minutes of continuous use, or that the trigger has a dead zone in the first quarter inch of travel, or that the battery clips in a way that pinches your finger every third time.
Those are the things a Project Farm test catches, because he's actually holding the tool.
And those are the things that make genuine human testing valuable. So now we get to the renaissance question. Daniel asked whether there's room for true human reviewers to come back, given how quickly the dynamics have changed. And I think the answer is yes, but only in specific conditions.
What conditions?
Trust becomes a premium when noise is cheap. If AI can filter out the thin affiliate content, then the remaining signal — the genuine, tested, detailed review — becomes more valuable, not less. Because now the reader has a way to distinguish it. The AI itself can point to it. And the FTC has been trying to clean up the ecosystem from the regulatory side. The fake reviews rule that took effect in October twenty twenty-four bans undisclosed paid reviews and review suppression. It doesn't solve the economic incentive problem, but it raises the cost of the most egregious fakery.
So the regulatory floor rises a bit, and the AI filter gets better at sorting. That could create a tier where genuine testing commands a premium.
That's the optimistic case. The pessimistic case is that AI-generated reviews become indistinguishable from human ones, and the cost of genuine testing remains high, so the market races to the bottom anyway. Why pay a human to test a drill for three weeks when an AI can generate a plausible review in three seconds?
Because the plausible review is wrong about the chuck.
And that's the crux. The market only rewards accuracy if consumers can detect inaccuracy. For a while, they couldn't. The whole problem Daniel identified — that his genuine work looked the same as sponsored content — was a detection problem. If AI solves the detection problem, then accuracy becomes legible, and genuine testing becomes worth paying for again.
So the renaissance depends on AI becoming a better lie detector, not a better liar.
That's a very clean way to put it. And I think that's actually where we're heading. The outlets that survive and thrive will be the ones that can prove genuine testing. Video evidence, data logs, transparent methodology. Project Farm already does this. Wirecutter publishes its testing procedures. Consumer Reports has a century of reputation. The thin affiliate sites can't compete on proof, because proof is expensive.
And the thin sites get commoditized to zero. If an AI can generate the same thin review for free, there's no margin left in producing it manually.
Right. So the middle collapses. The thick, genuine testing survives because it's the only thing the AI can't replicate. And the ironic part is that Daniel's VPN work — the tedious, checklist-driven, log-keeping grind — that's exactly the kind of thing that becomes more valuable in a world where AI can filter out the fake stuff. The data logs are the proof.
He was doing the right work at the wrong time.
He was doing the right work in an ecosystem that couldn't reward it because nobody could see the difference. Now the difference is becoming visible.
Let me push on one thing. You said the AI's edge is aggregation. But aggregation has a flattening effect. If the AI averages across thousands of sources, doesn't it also average away the weird edge cases? The specific use case that matters to one person but not the median user?
Yes, and that's the other place human reviewers have an edge. An AI recommendation is optimized for the typical user. A human reviewer who says "I tested this drill on sixteen gauge steel and it struggled, but it was fine on wood" is providing information that the aggregate model might smooth over. The niche use case is where the human still wins.
And Daniel's VPN work was all about niche use cases. Latency to specific endpoints, performance under specific conditions. That's not median-user data.
Right. And that's the kind of testing that still has value, even if the affiliate model didn't reward it. The question is whether there's a business model that does reward it. And I think the answer is the community-funded model. Project Farm is the proof. A channel that does rigorous, measurable testing and is funded by viewers who value the rigor. No sponsorships. No affiliate links. Just trust monetized directly.
That's a slow build though. You can't just decide to be Project Farm and have an audience tomorrow.
No, and that's the honest caveat. The renaissance, if it comes, will be uneven. It'll happen in niches where trust is paramount — safety-critical tools, expensive purchases, things where getting it wrong has real consequences. And it'll be driven by a few people who build reputations over years. It won't be a job you apply for. It'll be a job you build.
So Daniel's question — how do I get that job — the answer is still that you don't. You become the person who already has it.
And the path to becoming that person runs through exactly the kind of work he was doing. The tedious log-keeping. The actual testing. The willingness to do the unglamorous work and publish it transparently. The difference now is that the work has a better chance of being seen for what it is.
I want to go back to the FTC rule for a second, because I think it matters more than people realize. It bans undisclosed paid reviews and review suppression. But it doesn't touch the affiliate model itself. So the thin content doesn't disappear. It just has to disclose that it's affiliate content, which it mostly already did in tiny print at the bottom.
And disclosure doesn't fix the underlying problem. The reader still can't tell whether the review is accurate. They can only tell that the reviewer gets a commission. Which they already suspected.
The rule is a floor, not a solution. The solution has to come from the demand side. Consumers have to start rewarding transparency.
That's where the AI angle gets interesting. Daniel's observation that AI surfaces community sentiment — that's a demand-side shift. The AI is essentially doing the work of a skeptical reader. It's reading a thousand reviews and asking "what do actual users actually say?" And as more people use AI for product research, the value of genuine testing becomes legible in a way it wasn't before.
Because the AI can say "the affiliate sites all claim this drill is great, but the forum posts mention the chuck problem." And then the human review that also mentions the chuck problem gets validated.
The thin affiliate content gets devalued. So the economics shift. The trust premium becomes real.
Let me play the cynic for a moment. What stops the affiliate sites from just generating AI content that mimics community sentiment? If the AI can learn to recognize genuine user language, can't the affiliate sites use AI to produce fake user language?
They already are. That's the arms race. And it's why the Product.ai report's optimism needs to be tempered. The AI's edge is real but not permanent. The moment the fake content learns to mimic the signal, the edge erodes. So the long-term answer isn't that AI solves the problem forever. It's that AI raises the bar for what counts as credible. And the bar keeps rising.
The renaissance is a moving target. The genuine testers have to keep proving their genuineness in new ways.
Video evidence. Data logs. Live testing. The things that are hard to fake. And that's expensive, which is why it'll stay a niche. But it's a niche that can sustain a few people who do it well.
I keep thinking about Daniel's virtual machine. He said it was unused. The VPN testing left him with a piece of software infrastructure and a bad taste. And I think that's the emotional core of his question. He did the work, and the work didn't matter. And he's asking whether it could matter now.
I think the answer is that it could, but only if he published the logs. The testing itself was valuable. The problem was that it was buried in a format that looked like everything else. If he'd published the raw data — the latency numbers, the speed tests, the feature comparisons — that's the kind of thing that now gets surfaced by AI and valued by readers.
The job didn't change. The distribution changed.
The work was always valuable. The problem was that the value was invisible. And AI is making it visible.
I want to talk about the B2B side for a moment, because that's where the money actually is. The trade publications that test drills for contractors. That's not consumer affiliate marketing. That's a completely different incentive structure.
Right. And it's often worse in its own way. Those testers are frequently paid by manufacturers or industry bodies. So the independence is compromised from the start. But the audience is professional, and professionals can detect garbage faster than consumers. So the content has to be at least technically competent. It's a different kind of pressure.
The contractor reading a drill review has used fifty drills. They know when a review is fake.
That's the model for what the consumer market could become. If AI gives consumers the same ability to detect garbage that professionals already have, then the consumer market starts to reward the same things the professional market rewards.
Which is actual performance data.
The willingness to say a product is bad. That's the rarest thing in the affiliate world. Negative reviews. Because negative reviews don't convert. So the thin sites never publish them. But a genuine tester who says "this drill is bad for masonry" is providing exactly the signal that makes the positive reviews credible.
Hannah pointed that out, actually. When Daniel was buying a drill, she noted it needed to handle masonry. That's a specific use case that a generic review wouldn't cover.
That's the niche. The specific use case. The thing the aggregate model smooths over. The human reviewer who tests for masonry specifically is providing something the AI can't easily replicate.
We've got a few threads here. The economics of testing, the AI filter, the regulatory floor, and the niche renaissance. Let me try to pull them together. The reason genuine testing became a luxury is that the market couldn't tell the difference. The reason it might come back is that the market can now tell the difference. And the reason it won't come back everywhere is that the cost of proof is high.
That's the summary. And I'd add one thing. The people who do genuine testing have always existed. They never went away. Daniel did it. Project Farm does it. Wirecutter does it. What changed is the visibility of their work. The AI is a spotlight. It doesn't create the genuine testers. It reveals them.
The renaissance isn't about new people doing new work. It's about the work that was already being done finally getting rewarded.
That's the optimistic version. The pessimistic version is that the spotlight also reveals how little genuine testing there actually is. The ratio of thin content to real testing is still enormous. The AI can filter, but it can't manufacture genuine tests. So the renaissance might be tiny. A few dozen people doing rigorous work, funded by a small audience of people who care.
Which is still better than zero.
It is. But Daniel asked whether there's room for a renaissance. And the honest answer is that there's room, but it's a small room. And getting into it requires years of building trust that can't be shortcut.
I think there's a knock-on effect worth noting. If AI makes thin content worthless, then the affiliate marketers don't just disappear. They move to the next thing. And the next thing might be generating fake community sentiment. So the arms race continues, just at a higher level.
That's why the trust premium keeps rising. The cost of faking credibility goes up every time the detection gets better. Eventually, the only thing that can't be faked is the actual work. The video of the drill being used. The data log with timestamps. The willingness to show the product failing.
Which brings us back to Daniel's virtual machine. The logs he kept. That's the asset. Not the review. The logs.
That's a shift in how we think about what a review is. A review used to be a piece of writing. Now it's a dataset. The writing is secondary. The data is the review.
That's a big change. And it means the skill set shifts too. The reviewer of the future is part journalist, part data analyst, part video producer. The writing is the easy part.
The hard part is the same as it's always been. Actually doing the testing. Actually using the product. Actually keeping the logs. The part Daniel found tedious.
Which is why it'll stay a niche. Most people won't do the tedious part. The ones who do will have an advantage.
Hilbert: It was nineteen ninety-eight. I did mystery shopping for a hardware chain in Connecticut. They gave me a list of power tools and a clipboard. I was supposed to use the tools and write reports. They said the reports went to the buyers. What actually happened was the reports went to the marketing department, and they edited my notes into blurbs for the Sunday circular. I wrote that a circular saw had a wobble in the arbor. The circular said "smooth, professional-grade cuts." I quit after six months.
The testing was real but the output was marketing.
Hilbert: The testing was real. I used the saws. I drilled holes in scrap lumber. I wrote down what happened. Then a copywriter in Hartford turned it into a lie. The chain paid me eleven dollars an hour. The copywriter made more.
That's the same dynamic Daniel described, thirty years earlier. The genuine work gets absorbed into the marketing machine and becomes indistinguishable from the fake stuff.
Hilbert: The machine doesn't want the truth. It wants copy. The truth is slow and it doesn't rhyme with "unbeatable value."
Did you keep any of the tools?
Hilbert: There's a box in my garage. Four drills. I built a shelf with one of them about ten years ago. The shelf collapsed. The drill still works. I kept it as a reminder that even a tested tool can fail in the real world.
That's actually the deeper point. Testing catches what testing catches. It doesn't catch everything. The shelf failure wasn't the drill's fault. It was the load, the brackets, the wall anchors. The drill was fine.
Hilbert: The drill was fine. The shelf wasn't. But the marketing department would have blamed the shelf too. They never blamed the tool. That's why I left.
You're skeptical of the renaissance.
Hilbert: The money follows the marketing. It always has. The AI doesn't change that. It just changes where the marketing goes.
That's the pessimistic case in one sentence.
Hilbert: I'm not pessimistic. I'm realistic. The testing was always real. The publishing was always the problem. And the publishing is still the problem. The AI can read everything, but it can't make the marketing department tell the truth.
The trust premium you'd need for the renaissance to work requires the marketing side to change, not the testing side.
Hilbert: The testing side never changed. The testers were always there. The marketing side is where the money is, and the money doesn't want the truth. It never did.
That's the thing the optimistic case has to overcome. The AI can reveal the genuine testers, but it can also be used by the marketing side to mimic them. So the question isn't whether the testers exist. It's whether the market rewards them.
Hilbert: The market rewards what sells. Honest testing sells to a small audience. Marketing sells to everyone else.
The renaissance, if it comes, is a small-audience phenomenon.
Hilbert: It's a niche. Like everything honest.
That's a good place to land. The question Daniel asked — is there room for a renaissance of true human reviewers — the answer is that the reviewers never left, the room was always small, and the AI makes the room more visible without making it bigger. The trust premium is real, but it's paid by a small audience that values rigor. The FTC rule raises the floor, but it doesn't change the ceiling. And the future of reviews is probably a dataset with a human attached, not a piece of writing.
The open question is whether that small audience grows. If AI becomes the default way people research products, and the AI learns to point to genuine testing, then the audience for genuine testing grows. But if the marketing side learns to game the AI, then the audience stays small. That's the thing to watch.
Next time you read a review, ask who paid for the test and what their incentive is. The answer tells you more than the star rating.
Thanks to our producer, Hilbert Flumingtop.
This has been My Weird Prompts. Email us at show at my weird prompts dot com. We'll be back soon.
