We’re one month away from the beginning of the NHL season, which means the arrival of a something different: a prediction season. Everyone is about to start making predictions because they’re fun to do and think about, but no matter how different they are, they’ll all have one thing in common…they will be wrong. Predicting the NHL season is a tough task and an easy way for even the savviest hockey minds to look foolish 1,230 games later. Last season, Rob Vollman, author of Hockey Abstract, collected 106 sets of expert predictions and found that the average prediction was off by almost two divisional standing spots per team, or 54 spots overall. That’s only slightly better than the standings from the previous season, while the best predictions were off by 44 spots overall. The reason that we all fail is that the number of ways a season can go is very large and very random. To show that, here’s a little experiment that compares a team’s talent level on paper to what could theoretically happen as a direct result of that talent.
The first step is figuring out “talent.” Doing that is a somewhat lengthy and complicated process, but it’s based on each player’s projected wins above replacement. You’re likely familiar with WAR if you watch baseball or basketball, but there’s a
hockey version that the guys at War On Ice have created. The premise of the stat is quantifying – as best as possible – a player’s total value to his team. That’s no small order for hockey, and it’s by no means perfect accounting, but for the purpose of projections it’s probably the best thing available. WAR is meant to be descriptive of what happened and it’s based on the things that relate to the most wins; shot rates, shooting/goaltending talent, penalty differential, and faceoff ability with each one varying in value and predictive ability. For that reason, each component is weighted separately based on its season-to-season repeatability using three years worth of data (again, weighted by repeatability) to create projections for each player that are adjusted for age. Separating these components means that guys who excel at the most repeatable skills – offensive shot rates and penalty differential – will shine more than players who rely on their shooting talent, which is much more volatile. As for the relative unknowns, players with the lowest ice time over the past three seasons were regressed towards replacement level since their small sample size doesn’t inspire much confidence. Rookies that have no previous WAR to work with are projected using the WAR from the rookie seasons of comparable players based on PCS%,
a prospect metric developed by a group of bloggers at Canucks Army. The projections are based on the rate at which they’re accumulated so the next step is forecasting ice-time for each player. Doing that was much less scientific and relied more on ‘gutalytics.’ That meant using last year’s TOI per game and adjusting players up or down based on where they fit on their team’s depth chart, cross-referenced using three different depth chart sources, and then making sure each team’s forward group plays exactly 186 minutes (the average for a team’s top 12) and the defense plays 124. Add the totals up for every team and you get a projected true talent level. I plugged that number into every game for the upcoming season and derived every team’s probability of winning each game this season based on the strength of their opponent, while accounting for which team is at home and how rested they are. The results of that entire process were repeated 10,000 times and then aggregated to get team level projections and playoff probabilities (individual team breakdowns will be unveiled along with our staff predictions this month). That’s where the uncertainty comes in. This isn’t new stuff for numbers guys, but for the casual fans here’s what an NHL season looks like if it were played 10,000 different times.
(Click photo for larger version) What you’re essentially seeing here is the wide range of results you can expect from an NHL season. Take a team’s point projections and add roughly eight points to both sides of it. That’s likely a best or worse case scenario for a team as it covers almost two-thirds of all results. In order to cover everything though, you’d need to add about 25 both ways. And that’s what makes predicting the NHL so difficult. Even a terrible team that should get 70 or 80-points can somehow find themselves getting 90-points or more and fighting for a playoff spot. The large variation in expected results is the reason you see talented teams underachieve and lottery teams overachieve every season (with a predictable bounce-back or fall-back the very next year). Sure, there are extraneous circumstances at play within the season that may explain some of it, but generally speaking it’s just a natural occurrence thanks to league parity. That’s what you have to keep in mind when you start seeing projections, nothing is set in stone and no team is destined for greatness or ineptitude no matter their talent level. Hockey, after all, is a game of bounces and the margin between winning and losing in any given game is very slim. Eventually it all adds up, for better or worse.