the **spread** the (data) science of sports

Sun 18 May 2014

Rating and ranking

I recently picked up Who's #1?: The Science of Rating and Ranking, a really fun read on the many ways to take a list of items and order them by some score. Obviously, rankings are a huge topic of interest in sports, and my day job is working on recommender systems, so I saw this as the natural intersection of these things. The authors, Langville and Meyer, use college football as a running example throughout the text and I thought it would be a good exercise for myself and for readers if I worked through the various algorithms in the book using NFL data. The book requires a basic understanding of linear algebra, but don't let that stop you from reading.

I'll be using the recently released 2013 data from Armchair Analysis, but you could get this data just about anywhere.

[Sidenote: If you want to jump immediately to the technical end of things, I recommend you check out Sean Taylor's work on NFL rankings. In the spirit of this blog, I'm approaching this as an educational exercise instead of a finished product.]

Ratings and rankings are hugely important and sports fans are often
obsessed with finding out who's *really* number one. I'm going to use
the shorthand "ranking" for the rest of these posts, but you should be
aware that "rankings" are ordered lists and "ratings" are numerical
scores attached to those lists. Rankings give us an idea of the order,
ratings give us an idea of the magnitude of that ordering (i.e., *how
much* better is number one than number two).

**Massey's Least Squares Method**

The first method we'll be visiting is Massey's Least Squares Method,
which originated as part of Kenneth
Massey's undergraduate thesis. The basic
idea is that we can find some set of coefficients, *r, *that describe
the relative strength of a team. To compare two teams, *i* and *j*, you
simply subtract team *j*'s rating from team *i*'s rating to produce a
rough estimate of team *i*'s margin of victory.Massey's ranking work has
since moved on from this method (and he contributed to the BCS), but the
intuition behind this method is a good starting point.

I'll skip as many gory math details as possible, but this algorithm boils down into a simple formula:

**Mr = p**

where **M** is a 32x32 matrix (there are 32 teams), r is a vector of
size 32, and p, a vector of size 32.

**M**, being a square matrix, has the number of games played by each
team on the diagonal, and the negation of the number of times each team
played head-to-head on the off-diagonal spots.

For instance, if we sort the list of teams alphabetically by
abbreviation, we get the Arizona Cardinals in position 0 and the Atlanta
Falcons in position 1. That means the the (0, 0) cell of **M** would be
16 (the number of games played by the Cardinals) and the (0, 1 -- row 0,
column 1) cell of **M** would be a -1 because the Cardinals and the
Falcons played once in 2013.

Still with me? Good.

The *p* vector is just a list (in the same team order as the M matrix --
important!) of the end-of-regular-season point differentials for each
team (points scored - points allowed). For Arizona, this is 55 (for the
Denver Broncos, this is 207). So, our basic proxy for team strength is
point differential. We all know there are many problems with this, but
it's certainly part of the equation.

We need to solve for the last vector in that equation, *r, *which is the
set of coefficients that give us the mathematical relationship between
the teams and their score differentials. Anyone who's taken a statistics
class that uses linear algebra to teach regression might recognize the
above equation in a different form, **Xb = y** where X are our
covariates, b is the set of beta coefficients, and y is out dependent
variable. We're doing something very similar here, which is why this is
called the **least squares method**.

**Results - Overall Team Strength**

You can check Github for the gory details, but doing all of this math produces the list below. Nothing too counterintuitive emerges (though this is usually good for first steps). Seattle and Denver are the #1 and #2 teams. Jacksonville and Washington are the bottom two teams.

[table id=5 /]

**Offensive and Defensive Ratings**

We all know, however, that teams can be very different on opposite sides
of the ball. Massey quite rightly recognizes that point differentials
are composed of points scored and points allowed, so we should be able
to decompose our team strength scores into offensive and defensive
strength. In additional to decomposing the points, we'll need to
decompose our M matrix into its diagonal and off-diagonal elements. This
gives us to new matrices, **T** and **P **(again, check
Github
for the gory details).

We can use the ratings *r* that we computed above to find two new
vectors, *d* and *o*, by solving the following equation:

**(T + P) d = Tr - f**

This will produce *d*, the defensive ratings, and we can find *o* by
simply solving for it in:

**r = o + d**

*Offense*

[table id=6 /]

On offense, things start to look slightly less intuitive, though it's heartening to see that Denver is leaps and bounds better than the number 2 team, New England. Of course, these ratings are not really that useful if they're not predictive. That's a post for later in the series.

*Defense*

[table id=7 /]

Things look relatively reasonable! We've just constructed our first set of ratings and rankings. All of the code is up on Github.