A model that predicts the correct outcome nine times out of ten sounds strong until the tenth case is the one that mattered — a rare event the model almost never catches, hidden behind a headline accuracy number that looks fine because the other nine cases were easy. The confusion matrix, and the precision and recall numbers built from it, exist to catch exactly that kind of hidden failure in a classification model.
Reading a Confusion Matrix
A confusion matrix for a two-outcome prediction problem is a simple 2x2 table. One axis is what actually happened; the other is what the model predicted. Every prediction the model made falls into exactly one of four cells:
| Predicted positive | Predicted negative | |
|---|---|---|
| Actually positive | True positive (TP) | False negative (FN) |
| Actually negative | False positive (FP) | True negative (TN) |
A true positive is a case the model correctly flagged; a true negative is a case it correctly didn’t flag. The two error cells are where the useful detail lives: a false positive is something the model flagged that didn’t actually happen (a false alarm), and a false negative is something the model missed (a real case it failed to flag). Two models can have the identical overall accuracy while making very different kinds of mistakes — one all false positives, the other all false negatives — and the confusion matrix is what makes that difference visible.
Precision: How Trustworthy Is a Positive Prediction?
Precision answers a specific question: of every case the model flagged as positive, how many actually were? The formula is TP / (TP + FP) — true positives divided by everything the model called positive, correctly or not. A model with high precision rarely cries wolf; when it flags something, that flag is usually right.
Precision says nothing about what the model missed. A model that only ever flags the single most obvious case in a season and ignores every other real positive can still post perfect precision, as long as the one thing it does flag is correct — it simply isn’t being asked to catch much.
Recall: How Complete Is the Model’s Coverage?
Recall asks the complementary question: of everything that actually happened, how much did the model catch? The formula is TP / (TP + FN) — true positives divided by everything that was actually positive, whether the model flagged it or not. A model with high recall misses very little of what it’s supposed to be catching.
Recall says nothing about how many false alarms came along with that coverage. A model that flags almost everything as positive will have excellent recall — it’s hard to miss a real positive when nearly everything gets flagged — while its precision collapses, because most of those flags turn out to be wrong.
Why the Two Trade Off Against Each Other
Precision and recall pull in opposite directions because they respond oppositely to how aggressively a model flags something as positive. Making a model flag more cases as positive (lowering the threshold for a “yes” prediction) tends to raise recall, since more real positives get swept up along with the extra flags, and tends to lower precision, since a larger share of those flags are wrong. Tightening the threshold does the reverse: fewer flags, but a higher share of them correct. Neither direction is free — the choice is a trade-off, not an optimization with a single correct answer, and where to set it depends on what a false positive costs relative to what a false negative costs in the specific decision the model feeds into.
The F1 score — the harmonic mean of precision and recall — is a common way to summarize both in a single number when there’s no strong reason to weight one over the other. It only scores well when both precision and recall are reasonably strong; a model that’s excellent on one and poor on the other gets penalized more than a simple average of the two would suggest, which is the point of using the harmonic mean rather than the arithmetic one.
Confusion Matrices Beyond Two Outcomes
The 2x2 layout is the simplest case, but a confusion matrix generalizes cleanly to more than two possible outcomes — predicting play call as run, pass, or a designated-quarterback-run category, for instance, produces a 3x3 matrix instead of a 2x2 one. Each row is still an actual class, each column is still a predicted class, and the diagonal cells (row and column matching) are still the correct predictions; every off-diagonal cell is a specific kind of confusion — a pass predicted as a run, a designated run predicted as a pass — that a single accuracy number would blur together into one undifferentiated error rate.
Precision and recall extend to the multi-class case too, computed separately for each class by temporarily treating that one class as “positive” and every other class as “negative.” A play-type model might have strong precision and recall for predicting “pass” simply because passes are the most common call in most situations, while quietly performing far worse on the rarer “designated quarterback run” category — a gap that a single blended accuracy score across all three classes would hide, and that only checking each class’s own precision and recall separately reveals.
A Hypothetical Football Classifier
Suppose a model is built to flag plays likely to end in a turnover, and it’s tested against 200 plays from a hypothetical game sample where 20 actually ended in a turnover. If the model flags 30 plays as likely turnovers and 15 of those 30 really were turnovers, precision is 15/30 = 0.50 — half its flags are false alarms. If those same 15 correctly flagged plays represent 15 of the 20 real turnovers, recall is 15/20 = 0.75 — the model caught three-quarters of what actually happened, missing 5. That single set of numbers already tells a broadcast-graphics use case (which can tolerate false alarms to avoid missing real turnovers) something very different than it would tell a use case built around only surfacing high-confidence calls.
Real football classifiers built on this same logic already exist in the open sports-analytics ecosystem — nflfastR’s expected-pass model, for instance, estimates the probability that a given play will be a called pass based on down, distance, score, and time remaining, which can be turned into a hard yes/no classification (and evaluated with exactly this precision/recall framework) by applying a probability threshold to its output.
Computing It in Python
from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score
tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()
precision = precision_score(y_true, y_pred)
recall = recall_score(y_true, y_pred)
f1 = f1_score(y_true, y_pred)confusion_matrix returns the four counts directly; y_true is the actual outcome for each play or game, y_pred is the model’s hard yes/no prediction for the same set. Most classifiers output a probability rather than a hard label, which means the precision/recall trade-off described above is a choice made explicitly by picking the probability threshold used to convert that output into a yes/no call — not a fixed property of the model itself.
Frequently asked questions
What is a confusion matrix?
A confusion matrix is a table that breaks a classifier's predictions down by what actually happened versus what the model predicted. For a two-outcome (binary) problem it has four cells: true positives, false positives, true negatives, and false negatives — every prediction the model made falls into exactly one of them.
What's the difference between precision and recall?
Precision asks, of everything the model flagged as positive, how much of it actually was positive — it's a measure of how trustworthy a positive prediction is. Recall asks, of everything that was actually positive, how much of it did the model catch — it's a measure of how complete the model's positive predictions are. A model can score well on one while scoring poorly on the other.
Why isn't accuracy enough to judge a football prediction model?
Accuracy treats every correct or incorrect prediction as equally important, and on a lopsided outcome — a rare event like a specific type of turnover, or an uncommon play call — a model can score a high accuracy simply by predicting the common outcome every time, while completely failing at the rare outcome it was actually built to catch. Precision and recall, examined separately, surface that failure in a way a single accuracy number hides.
What is the F1 score?
The F1 score is the harmonic mean of precision and recall — a single number that only scores well when both precision and recall are reasonably high, and drops sharply if either one is weak. It's a useful summary when there's no strong reason to care more about one than the other, but it can obscure exactly which of the two is driving a low score, which is why the underlying precision and recall values are usually worth reporting alongside it.
How does the confusion matrix apply to a football prediction model?
Any model that outputs a yes/no call — did the offense convert this fourth down, did this pass get intercepted, did the favorite cover — can be evaluated with a confusion matrix once its predictions are compared against what actually happened. The four-cell breakdown shows not just how often the model was right, but which kind of mistake (missing a real positive, or falsely flagging one) it makes more often.
Should precision or recall matter more for a football model?
It depends entirely on what a wrong prediction costs in the specific use case. A model flagging likely turnovers for a broadcast graphic can tolerate some false alarms (lower precision) in exchange for catching more real ones (higher recall); a model used to make a single high-stakes call benefits more from precision, since a confident wrong call is costlier there than a missed one. There's no universally correct balance — it has to be chosen for the specific decision the model's output feeds into.