Mon 04 August 2014
Get in line.
People interested in sports analytics, particularly those wanting to become professionals, often wonder why teams don't invest more heavily in analytics. Teams, they argue, routinely throw good money after bad into garbage contracts and make unnecessary concessions of a million here and seven hundred thousand there. What gives?
This post was inspired by a tweet from Austin Clemens, the creator of absolutely fantastic shot charts at Nylon Calculus, which then sparked a lively NBA-centric discussion. Smart people with training in statistics, machine learning, and operations research, are willing to work for a fraction of what they could earn outside of sports. To top it off, people working in sports are routinely expected to work insane hours (sleeping at the office, working past midnight, etc.). If analytics is so valuable to these organizations, why aren't they paying market value? I'm going to argue that this argument is only partially true, but the true parts have some logical underpinnings. I only have experience with NFL teams, but I assume a lot of my thoughts are sport-independent. I'll offer them in sections.
Supply. The truth is that there is a glut of qualified people who want to work in sports. Depending on the sport you want to work in, there are usually only 30-32 teams at the top level. Most organizations only employ one or two analytics people. You do the math. An oversupply of overly qualified, highly interested applicants will drive salaries down, no matter what the industry. Employers aren't going to pay (much) over market value -- and their analytics people wouldn't be any good if they advised them to. One only has to look at the reaction to Andy Dalton's contract announcement today to see how much people have internalized the idea of market value.
You're giving up salary and job security to work for a team -- let's face it, most teams clean house every few years, and you're just one anti-numbers GM away from job hunting. On the other hand, you're getting to eat, sleep (sometimes), and breathe the game you love. That doesn't make it "right" -- many professions justify paying low wages because the profession is supposed to be a 'calling.' One only has to look at teachers' wages to see this. However, you are getting paid in many other ways. You have the ear (hopefully) of a coach or a GM. You get to see games from the booth or the sideline or the first row. You probably get to eat the majority of your meals for free, often with coaches and players. You probably don't pay for a lot of athletic gear or gym fees. There are perks.
It's not just analytics that faces this problem -- look at the entry level positions in coaching. An NFL quality control coach doesn't make much, and is expected to put in longer hours than many of the other assistant coaches. Of course, quality control coaches have the hope of climbing the career ladder. This path isn't so clear in analytics -- we can't all be Daryl Morey.
Assessment. It's extremely difficult for teams with little advanced analytics capacity to assess the quality of applicants. I spoke about this some previously in regards to the Sacramento Kings' contest to find draft analytics experts. An organization that has no advanced analytics capacity faces a cold start problem. How do they know who's good and who's not? The most likely outcome here is that an organization hires someone who is slightly more advanced than the most advanced person currently on staff. In the absence of some objective arbiter of talent, teams won't know if an applicant's work is mindblowingly complex and good -- or just garbage.
In these kinds of situations, signaling and credentials end up mattering a lot. This is why you see so many Ivy League grads and MBAs on analytic staffs. In the absence of information, you go with what seems to be an objective measure of quality. It's a rational, if potentially suboptimal, strategy. Unfortunately, most undergraduate and MBA programs don't actually prepare their students to think really hard about probability and statistics, so you get a lot of ad hoc analysis in Excel.
Results. The link between analytics and results is so tenuous, it's extremely difficult to measure, and it takes time. Even if you could somehow hold all else equal, most analytics exercises would produce small marginal gains (positive expected value). Sports are still noisy and stochastic affairs, and you're not going to be holding all else equal. It's just so difficult to demonstrate to a VP or GM the value added of paying someone six figures. Every hundred thousand dollars they pay their analytics staff is that much less wiggle room they have at the negotiating table with the players.
Then there's the fundamental problem of causal inference. My coworkers will know how much I love to bang on about this one. The problem in many cases is that history only runs itself once. You can't know what would have happened if you had done something differently. This is a very, very hard concept for people to understand (even highly trained statisticians forget it sometimes). Combine this with loss aversion, hindsight bias, and a host of other cognitive heuristics, and it's pretty easy to write analytics out of the picture.
This is a little easier to overcome in baseball than in other sports, and I suspect that's one reason there's been a 'revolution' there. In a sixteen-game season, how do you demonstrate that you added a third of a win? It's tough without buy-in from (high in) the front office.
Difficulty. Honestly, and it pains me to say this, a lot of what teams want from analytics just isn't that difficult. Just because you know all about convolutional neural networks or support vector machines doesn't mean those skills are necessarily going to be useful in sports. Many decisions come down to historical averages, expected value, and breakeven points. You don't need a PhD to figure those out, just a foundation in probability. That doesn't mean we can't improve, but just saying the skills don't dictate the salary -- the outputs do.
Variety. Related to the difficulty point, analytics means a lot of things to a lot of different people. There are people out there, who work for professional teams, that are doing absolutely garbage work. Other teams see the things that these professionals produce, and think "That must be analytics. Not worth much." It probably doesn't help that teams with losing records are more likely to invest in analytics to gain an edge, thus producing a real but spurious correlation between analytics talent and losing. This is why having a portfolio of work is as important as your ability to talk to people about what it means and what they should do differently because of it.
Canard. All of this said, I want to push back a little bit against the crux of the argument. Some teams are investing in analytics -- and they're paying near-market rates. It's not every team, and the teams doing so aren't advertising, but they're out there. We're in the infancy of the analytics movement in many sports. Give it time.