Sun 18 May 2014
Rating and ranking
I recently picked up Who's #1?: The Science of Rating and Ranking, a really fun read on the many ways to take a list of items and order them by some score. Obviously, rankings are a huge topic of interest in sports, and my day job is working on recommender systems, so I saw this as the natural intersection of these things. The authors, Langville and Meyer, use college football as a running example throughout the text and I thought it would be a good exercise for myself and for readers if I worked through the various algorithms in the book using NFL data. The book requires a basic understanding of linear algebra, but don't let that stop you from reading.
I'll be using the recently released 2013 data from Armchair Analysis, but you could get this data just about anywhere.
[Sidenote: If you want to jump immediately to the technical end of things, I recommend you check out Sean Taylor's work on NFL rankings. In the spirit of this blog, I'm approaching this as an educational exercise instead of a finished product.]
Ratings and rankings are hugely important and sports fans are often obsessed with finding out who's really number one. I'm going to use the shorthand "ranking" for the rest of these posts, but you should be aware that "rankings" are ordered lists and "ratings" are numerical scores attached to those lists. Rankings give us an idea of the order, ratings give us an idea of the magnitude of that ordering (i.e., how much better is number one than number two).
Massey's Least Squares Method
The first method we'll be visiting is Massey's Least Squares Method, which originated as part of Kenneth Massey's undergraduate thesis. The basic idea is that we can find some set of coefficients, r, that describe the relative strength of a team. To compare two teams, i and j, you simply subtract team j's rating from team i's rating to produce a rough estimate of team i's margin of victory.Massey's ranking work has since moved on from this method (and he contributed to the BCS), but the intuition behind this method is a good starting point.
I'll skip as many gory math details as possible, but this algorithm boils down into a simple formula:
Mr = p
where M is a 32x32 matrix (there are 32 teams), r is a vector of size 32, and p, a vector of size 32.
M, being a square matrix, has the number of games played by each team on the diagonal, and the negation of the number of times each team played head-to-head on the off-diagonal spots.
For instance, if we sort the list of teams alphabetically by abbreviation, we get the Arizona Cardinals in position 0 and the Atlanta Falcons in position 1. That means the the (0, 0) cell of M would be 16 (the number of games played by the Cardinals) and the (0, 1 -- row 0, column 1) cell of M would be a -1 because the Cardinals and the Falcons played once in 2013.
Still with me? Good.
The p vector is just a list (in the same team order as the M matrix -- important!) of the end-of-regular-season point differentials for each team (points scored - points allowed). For Arizona, this is 55 (for the Denver Broncos, this is 207). So, our basic proxy for team strength is point differential. We all know there are many problems with this, but it's certainly part of the equation.
We need to solve for the last vector in that equation, r, which is the set of coefficients that give us the mathematical relationship between the teams and their score differentials. Anyone who's taken a statistics class that uses linear algebra to teach regression might recognize the above equation in a different form, Xb = y where X are our covariates, b is the set of beta coefficients, and y is out dependent variable. We're doing something very similar here, which is why this is called the least squares method.
Results - Overall Team Strength
You can check Github for the gory details, but doing all of this math produces the list below. Nothing too counterintuitive emerges (though this is usually good for first steps). Seattle and Denver are the #1 and #2 teams. Jacksonville and Washington are the bottom two teams.
[table id=5 /]
Offensive and Defensive Ratings
We all know, however, that teams can be very different on opposite sides of the ball. Massey quite rightly recognizes that point differentials are composed of points scored and points allowed, so we should be able to decompose our team strength scores into offensive and defensive strength. In additional to decomposing the points, we'll need to decompose our M matrix into its diagonal and off-diagonal elements. This gives us to new matrices, T and P (again, check Github for the gory details).
We can use the ratings r that we computed above to find two new vectors, d and o, by solving the following equation:
(T + P) d = Tr - f
This will produce d, the defensive ratings, and we can find o by simply solving for it in:
r = o + d
[table id=6 /]
On offense, things start to look slightly less intuitive, though it's heartening to see that Denver is leaps and bounds better than the number 2 team, New England. Of course, these ratings are not really that useful if they're not predictive. That's a post for later in the series.
[table id=7 /]
Things look relatively reasonable! We've just constructed our first set of ratings and rankings. All of the code is up on Github.