Kount It!: The preseason is meaningless, isn’t it?

Imagine you’re on your way to a restaurant that you’ve never been to before, but you’ve heard nothing but great things about. They serve your favorite type of food, and you implicitly trust your friends’ recommendations, so you’ve been eagerly anticipating your visit for months.

You get to the restaurant, salivating and excited, and you’re heading inside when you suddenly notice out of the corner of your eye a few cracks in the sidewalk. Completely disgusted by the sight, you decide the restaurant’s food must be horrid if the sidewalk outside the restaurant is slightly damaged, and you turn around and make a beeline for the nearest Mighty Taco.

To me, that situation represents the basic equivalent of deriving any regular season conclusions from an NFL preseason game (aside from the Mighty Taco part; that’s just me being kind of hungry as I write this). Being an eternal pragmatist when it comes to the Bills, it’s not an attitude I have any problem adopting towards the preseason, and I know many Bills fans feel the same way. The fact that teams do not gameplan for preseason tilts (the 3rd preseason game aside), the heavy reliance on players who likely will not see much, if any, playing time once the regular season begins, and a general unwillingness to expose any star players to any sort of injury all contribute to this stance.

The Bills under Marv Levy made no bones about the fact that they didn't; care about winning in the preseason. (Photo by Ken Levine/Allsport)
The Bills under Marv Levy made no bones about the fact that they didn’t care about winning in the preseason. (Photo by Ken Levine/Allsport)

It may even be easier for us Bills fans to adopt this position than fans of most other teams, as the Bills under Marv Levy made no bones about the fact that they cared as much about winning in the preseason as some NBA players cared about staying in shape during the most recent lockout (I’m looking at you, Ray Felton), especially with regards to putting any of a relatively exotic offense on film for regular season opponents to study. More recently, the well-known fact that the Detroit Lions went 4-0 in the preseason before going 0-16 in the regular season in 2008 reinforced this attitude towards the meaningfulness (or lack thereof) of the NFL preseason.

With that in mind, I set out to quantify just how little the NFL preseason actually does mean with regards to regular season results. I’d be lying if I said this topic wasn’t something I wanted to investigate due to what I felt were some overreactions by both fans and media alike of the Bills’ preseason performance thus far this year. Not so much in an, “I told you so,” context, but instead an “It’s OK everyone, we can take a deep breath and relax,” context.

After looking at wins, losses, and point differential for both the preseason and regular season for all 32 teams over the last 10 years, I’m still of the mind that the preseason has very little bearing on what happens in the regular season, but there were some surprising facts that are very hard to ignore.

Before digging into the topic deeper, a few interesting facts about overall preseason performance and how it’s related to regular season/postseason performance over the last 10 years:

  • Only 5 teams have made the playoffs after going winless in the preseason
    • Two of those teams were the 2005 and 2010 Colts
    • 21 teams total have gone 0-4 in the preseason in the last 10 year
  • Only 2 teams had an overall preseason point differential worse than -45 and made the playoffs
    • Again, the teams were the 2005 and 2010 Colts, with -71 and -70 preseason point differentials, respectively
    • The Bills so far this preseason: -23
  • 9 of 19 teams that went undefeated in the preseason went on to make the playoffs
  • Out of the 142 teams that went on to have winning records in the regular season, 7 were 0-4 in the preseason

The Good

As one might expect, there is very little correlation between performance in the preseason and performance in the regular season. Though Bills fans (including myself), due to the Marv factor mentioned above, may have assumed that if there were any correlation between preseason and regular season performance, it’d be negative (i.e. the poorer a team performed in the preseason, the better it played in the following regular season), it was surprising to see that the minor correlations that do exist are, in fact, positive.

Over the last 10 years, there are slightly positive correlations between preseason point differential and regular season point differential, preseason win percentage and regular season win percentage, and preseason win percentage and making the playoffs. Interestingly enough, the strongest correlation I found was actually between points for in the preseason and win percentage in the regular season; about 2.5 times stronger than the correlation between preseason win percentage and regular season win percentage.

It’s very important to reiterate that these are slightly positive, which is to say that the numbers suggest that there’s a relationship between the two, but at such a small level as to be superficial in nature. This, of course, is good news for Bills fans, as the Bills’ performance throughout this preseason, while featuring some individual standout performances, have been woeful from a team perspective.

I do, however, think it’s important to note that the more successful teams over the past decade, including the Colts and the Patriots, downplayed the extent of the positive relationships found. Those two perennial playoff teams were regularly among the most severe outliers in terms of negative preseason performance and positive regular season performance over the past 10 years and again, I think Bills fans who remember the early 90’s have a special insight into how this might be the case: All were/are teams with prolific offenses and proven, established QBs who had a very large say in how the game played out on the field, even from a playcalling perspective; a rigid sense of continuity from year to year; and a general unwillingness to expose the intricacies of those offenses to regular season opponents to the extent that extremely “vanilla” offensive schemes were the flavor of their preseason game plans.

In the past ten years, only 5 teams have gone 0-4 in the preseason and made the playoffs. Tony Dungy's Colts did it twice. (Photo by Harry How/Getty Images)
In the past ten years, only 5 teams have gone 0-4 in the preseason and made the playoffs. Tony Dungy’s Colts did it twice. (Photo by Harry How/Getty Images)

With all that in common, it’s not hard to see how established teams may have a track record of performing more poorly than the rest of the league. This may be best evinced by the 2005 and 2010 Colts; in those two years the Colts went 0-4 in the preseason and made the playoffs (only 5 teams have accomplished that feat in the past 10 years and these Colts teams were two of them) and had horrendous preseason point differentials, but those preseasons were as representative of what those teams were capable of as much as Cheese Nips are representative of how good Cheez Its are. (For what it’s worth the one year the Patriots did go 0-4 in the preseason was also the season they went 11-5 in the regular season and missed the playoffs).

Finally, and the point I was dreading making most, as it implies I actually did pay some attention during school: Correlation does not equal causation.

This is to say that while there may be a small, positive relationship between the preseason and regular season, it would be folly to assume that the regular season performance was what it was because of the preseason performance. For example; there’s a positive correlation between the number of regular season games played and sales of Christmas trees, but the trees obviously aren’t being sold because of the NFL. It just so happens that the months leading up to Christmas coincides with the progression of the NFL regular season. It would take reams more data than I am able to mine from varying football websites to come anywhere close to determining if this is one of those coincidences or not.

The Bad

What good is looking at the entire preseason though, really? We all know that week 1 (and, for the Hall of Fame Game participants, the first two weeks) of the preseason is a time when teams play their starters for a series or two at most, and the rosters are at their fullest, brimming with over 30 players who won’t be with the team come the start of the regular season. Combine those facts with the knowledge that teams have, at most, a few weeks of training camp before that game, and it’s not hard to see that week 1 results may have a special degree of unimportance.

The same, of course, could be said for week 4. While we know by that time that most of the players who will play in the game will make the squad in the regular season, it’s commonly held tradition that teams will play their starters a very limited number of snaps, if any, and generally only have one goal in mind: Stay healthy. While this may make for games as viscerally exciting as a Terrence Malick film, it’s hard to debate the goal.

With that in mind, I narrowed down the preseason results I was looking at to just weeks 2 and 3. While many of the arguments that could be made to exclude week 1 of the preseason could be made to exclude week 2, simply comparing one week to regular season performance would narrow down our sample size, and week 2 is more analogous to week 3 (the “dress rehearsal” game in which most teams play their starters into the 2nd half) than any of the other weeks in the preseason.

When winnowing down the preseason stats this way, the correlations we saw earlier grow stronger, sometimes by 15%, but still remain small enough to be close to being considered inconsequential.

Nevertheless, it is the cause of some concern for Bills fans, as the Bills’ performance against the Vikings last week, while featuring more positives than the game against the Redskins, still wasn’t exactly confidence-building for many fans.

The Ugly

As mentioned earlier, comparing performance in weeks 2 and 3 of the preseason to regular season performance resulted in some slightly more positive correlations between regular season and preseason performance. This is of course to be somewhat expected as those games typically feature more gameplanning and substantive contributions from starters than weeks 1 and 3.

While those correlations may not represent the strongest in relationships, there are some factual tidbits that arise from this study that indicate it would be for the best for the Bills to put on a very strong performance this Saturday against the Steelers. For example:

  • No team in the last 10 years has had worse than a -38 point differential in games 2 and 3 of the preseason and still went on to make the playoffs
    • The team with the -38 point differential? You guessed it: Frank Stallone The 2005 Indianapolis Colts
    • The Bills are currently at -22 after last week’s loss to the Vikings
  • Of the 120 teams to make the playoffs in the last 10 years, only 11 of them lost both games 2 and 3
    • Two of those teams? You guessed it: Frank Stallone The 2005 and 2010 Indianapolis Colts
  • Only 2 teams have made the playoffs scoring less than 20 points combined in weeks 2 and 3 of the preseason

Still Skeptical

As damning as some of those stats may be (especially if the Bills were to lose badly this Saturday), I would again caution against reading too much into those factoids. I wouldn’t sit through the end of the game against Pittsburgh hoping that if we lose, we don’t lose by more than 16 (unless, of course, you have placed money on the Bills +16), and I still won’t sit here and say I think the overall team performance is more important than judging how players did in their one-on-one matchups on the field.

Being the third preseason game, it would seem the odds are more in the favor of the Bills doing well this year if they come out strong against Pittsburgh. (Photo by Hannah Foslien/Getty Images)
Being the third preseason game, it would seem the odds are more in the favor of the Bills doing well this year if they come out strong against Pittsburgh. (Photo by Hannah Foslien/Getty Images)

I will, however, admit that I’d like to see more out of both the offense and defense in the Pittsburgh game; not because I think it’ll automatically mean that there will be joy in Muddville this year, but because it would seem the odds are more in the favor of the Bills doing well this year if they come out strong against Pittsburgh.

Finally, let’s not forget the example that Marv’s Bills in the Super Bowl heydays set. One of those Bills teams, with a -45 point differential in games 2 and 3 of the preseason, broke one of the trends noted above. Another of those Bills teams went 0-2 in games 2 and 3 and still made the preseason, again going against the grain.

This just goes to show that there are exceptions to almost every rule. Much of this current Bills team, with its spate of late-round draftees and undrafted free agents, is a personification of being exceptions to rules: A 7th round QB from Harvard could never be an effective starter, a 7th round WR could never be among the league’s best, an undrafted RB who bounced from indoor football to Europe to NFL practice squads couldn’t be a major weapon. Is it too much to think that this team, resilient as they are, can’t overcome one more piddly little statistical “hurdle”?

What’s Coming for Kount It!

First and foremost, if you’ve liked what you’ve read in this column thus far, throw me a follow on Twitter @KoDon4. I try and spit out a few fun facts every now and then, along with other thoughts on the Bills, the NFL, and of course, #BillsMafia.

As the regular season (thankfully) hurtles towards us, I’ve been thinking of how best to keep the nature of this column going. I’ve signed up for the NFL all-22 game tape service in which I’ll have access to the coaches film of every game this year. I will not be analyzing each play and deciding who did what right and what wrong where; while I’ve seen enough of Jaws and Merril Hoge work NFL Matchup to think I might be able to see some things, let’s face it: Without knowing who is supposed to be where, and when, it’s difficult to state who may have been in position or out of position.

What I will be doing is logging stats that may not be regularly captured in NFL boxscores, including items like logging what defensive package the Bills show in certain situations, the spread offense, how often they show which look, and more. I mention this because I am more than open to suggestions from anybody about what they might like tracked during the season. Please send on your ideas to @KoDon4 and I’ll be glad to take the suggestions into account, and thanks for reading!