A crowdsourced football ranking is a points system wearing a statistics idea: many individual human ballots, aggregated by formula, resting on the same logic that once let a fairground crowd guess an ox’s weight to within 1%.
What “crowdsourced” means here, precisely
Every other method covered in this series — Massey least squares, the Colley matrix, a continuous-time Markov chain rating, Elo — computes a rating directly from game results using a fixed mathematical procedure, with no human judgment involved once the formula is set. A crowdsourced ranking works differently: it starts from many individual human orderings and combines them into one list through an aggregation rule. The AP Top 25 college football poll is the clearest, most public example of this second approach.
How the AP Top 25 aggregates its voters
The AP poll compiles a top-25 ranking from 62 sportswriters and broadcasters, each submitting an independent top-25 ordering. Points are awarded by position — 25 for a voter’s first-place team, 24 for second, continuing down to 1 point for twenty-fifth — and every voter’s points for a given team are summed across all 62 ballots to produce the final ranking. The poll dates back to 1936 for college football, with a preseason edition added in 1950, and individual ballots are made public, so any voter’s list can be checked against the aggregate result rather than trusted blindly.
This points-and-sum structure is itself a simple aggregation formula, not fundamentally different in spirit from the linear systems covered elsewhere in this series — the inputs are just human rankings instead of game results.
The statistical idea underneath crowdsourced rankings
The theoretical case for aggregating many independent judgments traces back further than any football poll. Francis Galton described a striking result in a 1907 article in Nature, “Vox Populi”: at a country fair in Plymouth in 1906, roughly 800 people entered a contest to guess the weight of a slaughtered, dressed ox. No individual guess was likely especially accurate, but the median of all 800 guesses — 1,207 pounds — came within about 1% of the ox’s actual weight of 1,198 pounds. Galton, no populist by temperament, found the result more creditable to aggregated judgment than he had expected.
The mechanism behind that result is straightforward: if individual errors are roughly random and not all biased in the same direction, averaging (or taking the median of) many independent estimates cancels much of that random error out, leaving an aggregate closer to the truth than most individual guesses. Applied to a football poll, the same logic suggests that 62 independent, moderately-informed opinions, aggregated by a points system, should land closer to “the truth” about team strength than any single voter’s opinion would.
Where the analogy to Galton’s crowd breaks down
The word “independent” is doing the heavy lifting in that mechanism, and it is also where a football poll differs meaningfully from a fairground weight-guessing contest. Poll voters are not isolated individuals guessing in a vacuum — they watch many of the same broadcasts, read overlapping commentary and rankings from other outlets, and are aware, to some degree, of what other voters and prior weeks’ polls have said. That kind of shared information and social awareness correlates voters’ judgments with each other rather than leaving them independent, which is precisely the condition Galton’s result depended on. A crowd whose members are quietly influencing each other is a different statistical object than a crowd of genuinely independent guessers, even if it still looks like “62 separate ballots” on paper.
This does not make a poll-based ranking worthless — it means the wisdom-of-crowds justification for it is weaker than the Galton anecdote alone suggests, and it helps explain a common, specific pattern in how polls behave: rankings tend to move gradually and cautiously week to week (a team rarely jumps 15 spots after one game) partly because voters are anchored to the previous week’s consensus, not building an ordering from scratch each time — a kind of correlation an algorithmic method built fresh from the full schedule graph each week does not share.
A more formal way to aggregate comparisons: the Bradley-Terry model
Where a points-based poll is a relatively simple aggregation rule, the Bradley-Terry model offers a more formal statistical structure for turning comparisons — human votes or head-to-head results alike — into a single ranked strength score. Introduced by Ralph Bradley and Milton Terry in 1952, the model estimates the probability that item i beats item j as
where p_i and p_j are latent strength scores for the two items being compared. Fit to a full set of pairwise comparisons, the model produces one strength score per competitor that best explains the entire pattern of who was preferred over whom — conceptually similar to how Massey least squares fits one rating per team to best explain a season’s margins of victory, but built from pairwise preference or win/loss data specifically, which makes it a natural fit for aggregating either human votes or game outcomes into a single ranked list.
What a Bradley-Terry-style aggregation would look like in practice
Applying the Bradley-Terry framework to a poll rather than to game results means treating each voter’s ballot as a set of pairwise preferences: if a voter ranks team A above team B, that ballot contributes one pairwise “vote” for A over B, regardless of how far apart the two teams sit on that voter’s list. Fitting the model to the full collection of pairwise votes across all voters and all pairs produces a single strength score per team — one that, unlike the AP poll’s simple points system, is estimated to best explain the entire pattern of pairwise agreement rather than summing position-based points directly.
The practical difference between the two aggregation methods shows up in edge cases. A points system rewards a team for being ranked highly by any voter, even if a large minority of voters leave that team off their ballot entirely — one voter’s #1 ranking is worth 25 points regardless of what the other 61 voters think. A Bradley-Terry-style fit instead weighs the full pattern of pairwise agreement and disagreement across every voter simultaneously, which can produce a different ordering for teams that most voters rank moderately but a few rank extremely high or extremely low. Neither approach is simply better; they encode different judgments about how much a single strong opinion should be allowed to move a combined ranking.
Crowdsourced and algorithmic rankings, compared
At the very top of a ranking, human polls and algorithmic ranking algorithms tend to agree closely — an unbeaten team with a clearly strong résumé is recognized both by human voters and by any reasonable formula. The two approaches diverge more in the middle of a ranking, where voter reputation, media narrative, and a single memorable win or loss can weigh more heavily on a human ballot than they would in a method built purely from the structure of the full schedule graph. Neither kind of ranking is simply “more correct” than the other by default — they are different instruments, one built from aggregated human judgment under Galton’s logic (with its independence assumption weakened by shared media exposure), the other built from a fixed mathematical rule applied uniformly to the same underlying results a poll voter also watched.
Frequently asked questions
What makes a football ranking 'crowdsourced' rather than algorithmic?
A crowdsourced ranking aggregates many individual human judgments — voters each submitting their own ordering — into one combined list, typically through a points system. An algorithmic ranking, by contrast, computes a rating directly from game results using a fixed mathematical procedure, with no human ballots involved at all.
How does the AP Top 25 college football poll aggregate its voters' opinions?
Sixty-two sportswriters and broadcasters each submit their own top-25 ordering. Each voter's first-place team receives 25 points, second place 24 points, down to one point for twenty-fifth; the points are summed across all voters, and the final ranking is sorted by total points. Individual ballots are made public, so any single voter's list can be checked against the aggregate result.
What is the 'wisdom of crowds' idea, and where does it come from?
It traces to a 1907 Nature article by Francis Galton, 'Vox Populi,' describing a contest at a 1906 country fair where roughly 800 people guessed the weight of a slaughtered ox. The median guess, 1,207 pounds, came within about 1% of the true weight of 1,198 pounds — a striking result given no individual guesser was likely that precise, which Galton took as evidence that an aggregated crowd estimate can outperform typical individual judgment.
Does the wisdom-of-crowds effect apply directly to football rankings?
The underlying statistical logic — that independent, moderately-informed estimates can average out individual error — is the same reasoning that supports poll-based rankings like the AP Top 25. But football rankings differ from Galton's ox-weighing contest in an important way: voters are not fully independent, since they often see the same broadcasts, read the same commentary, and are aware of each other's public reputations, all of which can correlate their judgments rather than leaving them independent.
What is the Bradley-Terry model and how does it relate to aggregating rankings?
The Bradley-Terry model, introduced by Ralph Bradley and Milton Terry in 1952, estimates the probability that item i beats item j as p_i / (p_i + p_j), where p_i and p_j are underlying strength scores. It provides a formal statistical way to convert a set of pairwise comparisons (or votes) into a single ranked strength score for every competitor, and is used well beyond sports, including in ranking systems built from human preference data.
Do crowdsourced rankings and algorithmic ranking algorithms usually agree?
At the top of a ranking, yes, most of the time — a clearly dominant, largely unbeaten team tends to be recognized both by human voters and by mathematical methods. In the middle of a ranking, human polls can diverge more from algorithmic methods, partly because voters weigh factors like a team's reputation or a single memorable win more heavily than a formula built purely from the full schedule graph would.