“What will probably happen” and “what should be done” sound like the same question answered from two angles, but they’re statistically distinct problems, and a model built to answer one doesn’t automatically answer the other. Fourth-down strategy is the clearest example in football, because it’s a situation where a team has genuine control over the decision, the possible actions are well defined, and decades of research have measured the gap between what the data recommends and what teams actually did.
Prediction: Estimating What Will Happen
A prediction model takes the current situation — down, distance, field position, score, time remaining — and estimates the probability of some outcome: will this drive end in points, will this team win the game. That’s the entire job. It doesn’t rank the available options, and it doesn’t say what a team should do differently; it describes the likely path forward given whatever is about to happen anyway.
Most of the modeling work covered elsewhere in this collection — win probability, expected points, rating systems — falls into this category. Those models are extremely useful precisely because they’re honest about their scope: they estimate what’s likely, and stop there.
Decision-Making: Comparing the Value of Available Actions
A decision-making model starts from prediction but adds a second layer: given the probabilities of different outcomes, and the value attached to each one, which of the available actions produces the best expected result. On fourth down, the available actions are concrete and limited — go for the first down, attempt a field goal, or punt — and each one leads to a different probability distribution over what happens next: a successful conversion continues the drive from a new down and distance; a missed conversion hands the opponent the ball at the spot; a made field goal scores three points and kicks off; a punt trades field position for a fresh set of downs for the opponent.
Turning that comparison into a single recommendation requires valuing every one of those branching outcomes on a common scale — which is exactly what win probability provides, since it converts field position, score, and time into one comparable number regardless of which action produced that situation. The action with the highest expected win probability, averaged across its own realistic range of outcomes, is the one a decision-making model recommends.
The Bellman Equation and Fourth-Down Strategy
David Romer’s 2002 NBER working paper, “It’s Fourth Down and What Does the Bellman Equation Say?”, is the foundational academic treatment of exactly this problem. Using play-by-play data from NFL seasons 1998 through 2000, Romer applied dynamic programming — a method built around the Bellman equation, which values a decision today in terms of the best decisions still available afterward — to estimate the expected value of possessing the ball at every field position, then combined those estimates with the actual observed results of kicks and conventional plays to calculate the expected payoff of going for it versus kicking in a wide range of fourth-down situations.
The paper’s central finding was blunt: teams’ actual fourth-down decisions departed systematically, and by a statistically large margin, from what the dynamic-programming analysis calculated as the better choice. Coaches were kicking and punting in a large share of situations where the analysis found that going for it produced a meaningfully higher expected value — evidence of a gap between a decision-making calculation and the decisions actually being made on the field.
Why the Gap Existed — and Has Partly Closed
The gap Romer documented wasn’t a prediction failure — coaches generally understood the probability of converting a given fourth down reasonably well. It was a decision-making failure: a tendency to weigh the visible, immediate cost of a failed fourth-down attempt (the crowd reaction, the field position handed to the opponent, the accountability of having “gone for it and failed”) more heavily than the statistical value calculation supported, relative to the quieter, harder-to-see cost of consistently punting away situations with a positive expected value.
That gap has narrowed over time, though not closed entirely. Tracking built on the open-source nfl4th model — which estimates a go_boost metric, the expected win-probability gain or loss from going for it relative to the next-best alternative, using game state, time, and score along with win-probability inputs — shows coaching decisions moving closer to the model’s recommendations across the 2014–2020 seasons: in situations the model recommended clearly going for it, actual go-for-it rates rose over that period, though coaches remained more conservative than the calculated recommendation in a meaningful share of borderline cases.
Decision-Making Requires a Value, Not Just a Probability
The reason a prediction model can’t be turned into a decision-making model just by comparing two probability numbers is that raw probability doesn’t account for how the different actions’ outcome distributions differ in shape, not just in likelihood of success. Going for it on fourth-and-short and missing costs a team the ball at roughly the current spot; punting instead trades a chance at that outcome for a guaranteed, smaller field-position loss. A fair comparison has to weigh the full range of each option’s outcomes — success and failure alike, and how costly each failure actually is — not just the headline conversion percentage, which is exactly what a win-probability-based expected-value calculation, like the one behind Romer’s analysis, is built to do and a bare prediction model is not.
This is also where career incentives plausibly explain part of the gap Romer measured, separate from anything in the statistics themselves: a coach who goes for it on fourth down and fails faces a more visible, more immediately attributable kind of criticism than a coach who punts in a situation that a value calculation says was actually worse. Romer’s paper focuses on measuring the size of the gap between actual decisions and the calculated optimum, not on explaining coaches’ motives, but the asymmetry between a decision-making calculation’s verdict and the more visible, personally costly failure mode of an aggressive call is a plausible reason the gap persisted for as long as it did even after the underlying research became widely available.
Why This Distinction Matters Beyond Fourth Down
Fourth-down strategy is the clean textbook case because the decision points are discrete and well-defined, but the same prediction-versus-decision distinction applies anywhere a model’s output is meant to inform a choice rather than just describe a likelihood. A model that reports “this team has a 38% chance of winning” is doing prediction. A model that says “given that 38% and the specific value of each remaining outcome, this particular play call maximizes expected win probability” is doing decision-making, and it requires an explicit value structure — what each outcome is actually worth — that a pure prediction model was never built to supply. Conflating the two, and treating a prediction model’s output as if it already contained a recommendation, is a subtler version of the same gap Romer’s research measured in coaches’ actual fourth-down calls.
Frequently asked questions
What's the difference between prediction and decision-making in statistics?
A prediction model estimates what is likely to happen — the probability of an outcome given the current situation. A decision-making model goes a step further: given those probabilities and the value of each possible outcome, it calculates which available action produces the best expected result. Prediction answers 'what will probably happen'; decision-making answers 'what should be done about it.'
Why can't a good prediction model be used directly to make decisions?
A prediction model estimates probabilities but says nothing about how much each outcome is worth. Two situations with identical win probabilities can call for opposite decisions if the value of success and the cost of failure differ between them — a model built only to predict outcomes has no way to represent that difference unless it's explicitly extended to weigh outcomes against each other.
What did David Romer's fourth-down research find?
Romer's 2002 study used play-by-play data from NFL seasons 1998–2000 and dynamic programming (the Bellman equation) to estimate the expected value of going for it versus kicking on fourth down in various field positions. The paper found that actual coaching decisions departed systematically, and by a statistically large margin, from the decisions the analysis identified as preferable — teams were kicking or punting far more often than the value calculation supported.
Have NFL coaches changed their fourth-down decisions based on this kind of research?
Yes, over time. Tracking using the open-source nfl4th model shows that the share of situations where the model recommended going for it and coaches actually did so rose across the 2014–2020 seasons, indicating coaching decisions have moved closer to the model's calculated recommendations, though not all the way to matching them in every situation.
What is the Bellman equation, and why does it matter for football decisions?
The Bellman equation is the core tool of dynamic programming, a method for solving decisions that unfold in stages, where the value of an action today depends on the expected value of the best decisions still to come. Applied to football, it lets a model value possessing the ball at a given field position not just by what happens on the current play, but by the full sequence of decisions and outcomes that follow from it.
Is a fourth-down decision model the same thing as a game-outcome prediction model?
No — they're built for different questions and often share only their underlying inputs. A game-outcome prediction model estimates the probability of winning given the current state. A fourth-down decision model uses that same kind of win-probability estimate as a building block, but adds a comparison across the available options (going for it, kicking a field goal, punting) to recommend which one maximizes the team's expected win probability.