The AI win your team reported last quarter is probably not real.
That isn't a knock on your team. It's a measurement problem. Most teams, and plenty of so-called AI experts, have no solid way to tell whether an AI feature is actually working. They have a demo that looked good and a number that went up. That is not the same thing.
One question tells you, in about five minutes, how real that win was. This post covers that question, why the usual answers fall short, why public benchmarks don't fix the problem, and what real AI evals look like.
The One Question to Ask About Any AI Win
Ask your team: how do you know this is actually working?
Another way to put it: how did you measure the quality of the output?
Listen closely to the answer. You'll often get something that sounds confident but never explains how anything was measured. Dig one level deeper. In most cases the answer comes down to one of two things.
- Vibes. Someone looked at the output and said it looks good, or this version looks a little better than that one.
- LLM as a judge. One AI model grades the output of another.
If either of those is the final answer, you don't know whether the win is a win. It could be a disaster waiting to happen.
Why Vibes and LLM as a Judge Both Fail AI Evaluation
Vibes are obviously a bad measure. One engineer's gut feel isn't repeatable and doesn't scale. It can be useful very early, when an engineer is getting a feel for the problem. It is not a final result.
LLM as a judge sounds more rigorous. It isn't. It is incredibly unreliable. Give it the exact same text in a different order and it can give you a different answer. Minor tweaks to how you present the input can flip the verdict.
A measuring tool that changes its reading when you rearrange the input isn't measuring much.
Both are fine as a starting point. Neither should be what you lean on when you decide a feature is ready. Real quality measurement goes deeper than one engineer's opinion or AI judging itself.
Why AI Benchmarks Don't Answer the Question Either
When teams can't measure quality themselves, they fall back on public benchmarks. The model scored well on the leaderboard, so it must be good.
There are two main ways models get graded. Benchmarks grade them at a very high level. Evals grade them on a specific use case, if evals get done at all. Benchmarks get the headlines. Evals are what matter to a business.
Benchmarks have three big problems.
They measure a tiny slice
A benchmark measures a very narrow slice of what these models can do. Maybe one tenth of one percent of one percent of the ways a business could use them. A strong score on that slice tells you very little about your workflow.
Companies train on them
Benchmarks are public, and model companies train on them. Even when a benchmark is hard to train on directly, companies optimize their models for it. A better score gets them funding, attention, and press. Once that happens, the benchmark loses a lot of its meaning, and fast.
Small gaps mean almost nothing
When two models score close together on a benchmark, that doesn't mean the higher one is better. Not even at that specific task. The numbers are too close to take much meaning from.
Benchmarks do carry some signal at the extremes. If every model sits near zero and one jumps to a non-trivial score, or one model scores far above everyone else, that tells you something. Same when a benchmark gets maxed out. Even then, it's a narrow measure inside a narrow measure inside a narrow measure.
There Is No Best AI Model
Benchmarks feed a bigger myth: that there is one best model. People ask which model is the best right now as if there's a clean ranking. It reminds me of the anime I watch with my son. Every hero has a rank. Number one beats number two. Number two beats number three.
That's not how the world works, especially with technology. There may have been a time when one model really was best at nearly everything. That time has long passed. Even the model with the best overall benchmark scores still loses to other models on quite a few things.
Every system has trade-offs, and training is no different. When you train a model to be better at one thing, it often gets worse at other things. When GPT-5 came out, many people preferred GPT-4o. For certain things, they were right.
My own testing backs this up. For our paper The Myth of Ever-Faster AI, we evaluated models from 2022 to 2026 on real production work. On the 25 task types in the paper's main chart, a model released before 2026 matched or beat the best 2026 model on 16 of them. GPT-3.5, from 2022, tied it on four.
If a model that old keeps up on a quarter of real tests, the claim that newer models are objectively better across the board doesn't hold.
Custom evals turn up results like this all the time. Lighter models sometimes beat more expensive ones. Older models sometimes beat newer ones. You won't see that on a leaderboard. You'll see it when you test your own use case.
What Real AI Evals Look Like
The fix is to get a bit creative and score output with code-based, deterministic methods. Deterministic means non-random. Same input, same score, every time.
One example is vector embeddings. Embeddings depend on a model, but they are deterministic. Put in the same text and you get the same output every single time. That lets you judge how relevant a result is in a consistent, reliable way.
For each output I want to measure, I build about three to seven scoring metrics and run every output through them. The right metrics depend on the task. The principle doesn't change: measure quality with tools that give you the same answer twice.
Then evaluate models on what you actually want to use them for. If your use case is complicated enough, split it into individual tasks and use whichever model is best at each one. That only works if you run high-quality evals for every task. This matters most in a business.
I go into more detail in The Myth of Ever-Faster AI.
Your Next Step
Go back to the last AI win your team reported. Ask how they measured the quality of the output. If the answer is vibes or another model's opinion, treat the win as unproven until it's measured properly.
Frequently Asked Questions
What is the fastest way to tell if an AI win is real?
Ask your team how they measured the quality of the output. If the answer is vibes or LLM as a judge, the win is unproven.
Why is LLM as a judge unreliable?
It can give a different answer when you give it the exact same text in a different order. Minor tweaks to the input can change the verdict, so it should not be your final measure of quality.
Are AI benchmarks useless?
Not completely. They carry some meaning when a model jumps from near zero to a real score, or when a benchmark gets maxed out. But they measure a tiny slice of use cases, companies train on them, and small score gaps mean very little.
Is there one best AI model?
No. Training a model to be better at one thing often makes it worse at others. In the 25 task types charted in 42 Robots AI's paper The Myth of Ever-Faster AI, a model released before 2026 matched or beat the best 2026 model on 16 of them.
How should a business evaluate AI output quality?
Build custom evals for your own use cases. Use deterministic, code-based scoring such as vector embeddings, with several scoring metrics per output, and evaluate each task separately.
At 42 Robots AI we build AI evaluations that measure quality on your real use case. If you want a second opinion on an AI win, let's talk.