When the Vikings hired Mike Zimmer, I started following the team again. They improved quickly from pathetic to respectable. I searched for football news that wasn’t just gossip or marketing to help me understand what I was watching.
I found Pro Football Focus (PFF). They didn’t just take available statistics and crunch the numbers. They built their own foundation by watching each player on each play, assigning a grade, and then applying analytics with that information.
There are parallels between the lessons of football analysis and the lessons of analysis for our culture. They are outlined below. The lessons are in bold. The context here is the NFL--but can you see how they might be lessons for American culture as well?
PFF tries to be objective, but that doesn’t mean they don’t make decisions. Any decision process can be criticized as biased. That does not mean every decision process is equal.
PFF makes judgments about how good or bad a play is. They judge every player on every play. They test inter-rater reliability. They ask NFL coaches and former players to review their work. They have a rigorous process.
Their results are not perfect, but they have an impressive track record. NFL coaches have responded by criticizing PFF and other analytical approaches. Why?
People with control have a tendency to be defensive and discount things that threaten their control. That’s what is going on when coaches and GM’s discount what PFF has to say. Coaches have had a monopoly on much of the information needed to make good coaching decisions and they don't want to lose that control.
It’s easy for coaches to attack the analytics partially because football is a high-variance game. For starters, the ball is not round. When a player fumbles, the ball bounces unpredictably. Those unpredictable bounces can determine who wins or loses.
But no matter how talented a running back is, he doesn’t impact the game as much as a quarterback does (this is another insight PFF has driven home). Throwing the football is more valuable than running it. Adrian Peterson could never average as many yards running the ball through the defense as Brett Favre averaged by throwing the ball over the defense.
That’s why Favre appeared in two Super Bowls and Petersen didn’t appear in any. It’s why Petersen never even made a deep playoff run until he had an excellent quarterback on his team--none other than Brett Favre. Favre didn’t choose to be gifted at quarterback rather than running back--that was luck. Peterson didn’t choose to be gifted at running back instead of quarterback--that was also luck. Hard work and talent matter, but people underestimate the impact of luck.
It's not just luck. PFF identified that quarterbacks (QB’s) sometimes get credit for things they do not earn. One screen pass might go 50 yards for a touchdown; on another the receiver drops the ball. A QB with great receivers looks confident and a QB with receivers who can't get open looks feckless, although the actual QB skill may be similar. But we tend to glorify individual performance and underestimate the impact of the supporting cast.
I grew up with football. As a kid, I spent hours and hours in the field across the street pretending to be Ted Brown or Ahmad Rashad while my friend was Tommy Kramer. I played High School football and had a blast doing it. I’ve followed the Vikings on and off, cheering as Randy Moss grabbed footballs out of the air and Adrian Peterson busted loose for big runs. As irrational as it is, they just feel like my team.
I found Pro Football Focus (PFF). They didn’t just take available statistics and crunch the numbers. They built their own foundation by watching each player on each play, assigning a grade, and then applying analytics with that information.
There are parallels between the lessons of football analysis and the lessons of analysis for our culture. They are outlined below. The lessons are in bold. The context here is the NFL--but can you see how they might be lessons for American culture as well?
PFF tries to be objective, but that doesn’t mean they don’t make decisions. Any decision process can be criticized as biased. That does not mean every decision process is equal.
PFF makes judgments about how good or bad a play is. They judge every player on every play. They test inter-rater reliability. They ask NFL coaches and former players to review their work. They have a rigorous process.
Their results are not perfect, but they have an impressive track record. NFL coaches have responded by criticizing PFF and other analytical approaches. Why?
People with control have a tendency to be defensive and discount things that threaten their control. That’s what is going on when coaches and GM’s discount what PFF has to say. Coaches have had a monopoly on much of the information needed to make good coaching decisions and they don't want to lose that control.
It’s easy for coaches to attack the analytics partially because football is a high-variance game. For starters, the ball is not round. When a player fumbles, the ball bounces unpredictably. Those unpredictable bounces can determine who wins or loses.
Bad teams can beat good teams because of luck. Coaches have a lot of variance to hide behind. Coaches can always point out exceptions to a rule or idea. While exceptions to the rule are an effective distraction, over many instances or over long periods of time, even high-variance situations follow a set of rules.
Rules like what? The New York Times started operating a “4th down bot” that made judgments about whether teams should “go for it” on fourth down. It turns out that teams, on average, did not go for it nearly enough.
It’s taken years, but we’re finally seeing coaches start to follow the advice. First, coaches had the usual criticisms: “You just don’t understand football.” Then some coaches who were more aggressive on fourth down started winning more close games. It's hard to hide the results, because the games are on display.
Rules like what? The New York Times started operating a “4th down bot” that made judgments about whether teams should “go for it” on fourth down. It turns out that teams, on average, did not go for it nearly enough.
It’s taken years, but we’re finally seeing coaches start to follow the advice. First, coaches had the usual criticisms: “You just don’t understand football.” Then some coaches who were more aggressive on fourth down started winning more close games. It's hard to hide the results, because the games are on display.
The NFL is a spectator sport, so the product and much of the process is easy to observe. Things that have to withstand public scrutiny almost always produce better results in the long run. Coaches can be demure about their reasoning, but the decision and its consequences cannot be hidden.
Even when the results cannot be hidden, we can be fooled by our biases. For example, when we see something dramatic or impressive, we tend to assume it is effective. Coaches see running backs do impressive things and believe that those impressive things must make an impressive difference in the outcome of football games. Impressive things are often not effective things, and effective things are not always impressive.
But no matter how talented a running back is, he doesn’t impact the game as much as a quarterback does (this is another insight PFF has driven home). Throwing the football is more valuable than running it. Adrian Peterson could never average as many yards running the ball through the defense as Brett Favre averaged by throwing the ball over the defense.
That’s why Favre appeared in two Super Bowls and Petersen didn’t appear in any. It’s why Petersen never even made a deep playoff run until he had an excellent quarterback on his team--none other than Brett Favre. Favre didn’t choose to be gifted at quarterback rather than running back--that was luck. Peterson didn’t choose to be gifted at running back instead of quarterback--that was also luck. Hard work and talent matter, but people underestimate the impact of luck.
It's not just luck. PFF identified that quarterbacks (QB’s) sometimes get credit for things they do not earn. One screen pass might go 50 yards for a touchdown; on another the receiver drops the ball. A QB with great receivers looks confident and a QB with receivers who can't get open looks feckless, although the actual QB skill may be similar. But we tend to glorify individual performance and underestimate the impact of the supporting cast.
As you can tell, I’ve enjoyed listening to the analysis of PFF. They may love football, but their job as analysts helps them see the game with relative objectivity. I use the term relative because they can’t be perfectly objective, but people really can be more objective when it’s their job to do so, and when they are not beholden to one side or another.
I've praised PFF at the expense of coaches, but coaches have great value as football experts. Each player on a football team has to learn hundreds of things about their position in order to do it at that high level. A football game is very complex and requires intense orchestration. Coaches still routinely out-coach opponents and win games because of it. Expertise still matters, and it matters a lot.
I have a few more observations, but they are less analytical and more personal.
I grew up with football. As a kid, I spent hours and hours in the field across the street pretending to be Ted Brown or Ahmad Rashad while my friend was Tommy Kramer. I played High School football and had a blast doing it. I’ve followed the Vikings on and off, cheering as Randy Moss grabbed footballs out of the air and Adrian Peterson busted loose for big runs. As irrational as it is, they just feel like my team.
My fealty to the Vikings is a perfect example of the pervasive influence that bias and identity have on our choices. I watch a game with a Packers fan, and we come to consistently different conclusions about whether a catch was a catch or a penalty was a penalty or a touchdown was a touchdown. Packers fans thought Anthony Barr’s hit on Aaron Rogers a few years ago was uniquely dirty, while Vikings fans thought it looked just like lots of Clay Matthews sacks.
Objectively, that hit was brutal. I know that football is a brutal sport, and I do not want my son to play the game. Despite the league’s efforts to make the game safer, concussions (among other injuries) are a routine part of fair play, and as players and coaches like to remind us, “The injury rate in the NFL is 100%”. So why don't I just pick a different sport?
Behavior is slow to follow moral understanding, especially where tradition or social pressure is at play, and I am no exception. Some part of me feels that if I leave “my team”, I’m losing something I’ve invested in. I continue to invest in something that has questionable return on investment, and even runs counter to some of my values.
I think these things about the NFL are true. I think they apply to aspects of our culture as well. Culture is so important, it deserves some scrutiny. But it’s much easier to talk about a game.
Comments