AI football predictions: how to tell a real model from a mystery box

"AI football predictions" is now a marketing sticker as much as a technology. Some of what wears it is genuine modelling; a lot of it is a black box with a confident voice and a screenshot of a good week. If you're going to trust a model's read, you need a way to tell the two apart. Here are the seven questions that do it.

1. Does it show a distribution, or just a pick?

A real model produces a probability for every outcome — home/draw/away, every scoreline, over/under. A mystery box hands you a single answer ("back 2–1") with no sense of how likely it is. If you can't see the odds behind the pick, there's nothing to evaluate. Football's most likely scoreline still usually has under a 15% chance — a model that hides that is hiding the whole point.

2. Is it calibrated — and can it prove it?

The professional test isn't accuracy, it's calibration: over hundreds of matches, do the things it calls 60% actually happen about 60% of the time? A trustworthy model publishes a reliability curve and a Brier score and lets you check. A mystery box quotes one accuracy number and no method. We wrote up why calibration beats accuracy in how we model a match.

3. Was it tested out-of-sample?

Fit any model hard enough to the past and it looks flawless — on the past. The only honest test is performance on matches it never saw during fitting. Ask "how does this do out-of-sample, over how many games?" If the answer is a hand-picked hot streak, it's a marketing artefact, not evidence.

4. Does it explain where its numbers come from?

You don't need the source code, but you should be able to learn the shape of the method: what inputs it uses (goals, xG, the market), how it weights recent form, how it handles a promoted side with little history. A model that can explain itself in plain language (like this) is a different animal from one that just says "our AI decided."

5. How does it treat the market?

The closing betting line is one of the best public forecasts that exists — genuinely hard to beat. A serious model respects it: it uses the de-vigged market as an input or a benchmark and reports where it disagrees and by how much. A model that claims a huge edge on every game, ignoring the market entirely, is telling you it hasn't met the market yet.

6. Does it admit uncertainty?

Real modelling is full of honest hedges: small samples, injury news that lands late, the irreducible noise of a low-scoring sport. A model that surfaces those limits is more trustworthy, not less. Guarantees and "locks" are the tell of a sales page, not a forecast.

7. Is it selling a read or selling certainty?

This is the one that catches most of them. A genuine analysis tool gives you a quantified view and lets you decide. A mystery box sells confidence — the feeling of knowing — because certainty converts better than probability. If the pitch is profit guarantees and exact scores, you're being sold the one thing no model can deliver.

The takeaway

You can't see inside every model, but you can interrogate its behaviour. Distribution, not a pick. Calibration, not accuracy. Out-of-sample, not a hot streak. Respect for the market, honesty about uncertainty, and a read rather than a guarantee. Score a service against those seven and the mystery boxes fall away fast.

That checklist is, deliberately, the spec Momus Modal is built to. A quant model reads every match into a full, calibrated grid, and Momus writes up the read — analysis, not tips. See what that looks like on the desk.