James Mirtle has a series of pieces in the Globe on the new hockey analytics: “The Moneypuck Revolution“, “Why Haven’t Advanced Stats caught on in the NHL“, and “Hockey’s New Numbers“. I think he was very clever to use the movie “Moneyball” to pitch a series he’s wanted to do. I am also happy to hear that Gabe Desjardins is making some money advising some NHL teams. I am sure he is worth it.
James knows me well enough to know that I’m offside on this issue. His series prodded me to address the issue in some depth. (Fair warning – this isn’t going to be short.) I even went over to the Sloan Institute that he referenced and watched the panel discussion on the subject. (It was an interesting, if inconclusive discussion.)
For the record, I’m not against people looking at any of this stuff. I played with it myself for quite a while in the 1980′s and then again in the late 1990′s. (When I was working for About.com, Dean Oliver was the basketball guy and we had several discussions about the subject.) The data has come miles since those days, but none of it solves the fundamental problem.
Hockey numbers are not useless. We can learn things with the statistics. We have already learned that hockey is heavily influenced by randomness, a fact that puts lie to most of what passes for hockey punditry. In fact, that’s the best argument for hockey analytics – the numbers are real, if often misleading, and lots of hockey narratives are fantasy. Hockey analytics are increasing in popularity at least partly because the usual alternative – media mythmaking – is worse. Stats can be used to skewer the ridiculous claims made by teams and the reporters who follow them. (See Tyler Dellow.) Stats can also help identify when a player (or team) result has been heavily influenced by luck. Finally, I’m intrigued by statistical comparables. (At age 22, Joe has played so many games and done this and that. Here is a list of all the players who did that well that young. The worst of them is pretty damn good. Therefore Joe will probably be pretty damn good. I like statistical comparables because a lot of my subjective evaluations are based on human comparables. Ryan Kesler is Bob Gainey with more offense. Jonathan Toews has a young Steve Yzerman skillset. Sidney Crosby is Peter Forsberg plus, and so on.)
Hockey analytics are not useless.
Hockey statistics will never do what we want them to do, which is to effectively evaluate individual hockey players. To give us answers when considering a trade or a personnel decision. To tell us whether this third line winger creates more wins than that number four defenseman. Baseball statistics can produce answers to these kinds of questions, while hockey statistics can only produce more questions.
Why? Because baseball statistics describe what actually happens in baseball games. The stats add up to runs and to wins. Hockey statistics do not add up to goals. Goals or proxies for goals like shots, shots and attempted shots or even quality shots, underpin all the analyses. None of these statistics say anything about how the scoring chance, the goal was achieved. The actual activities that go into creating the chance are not recorded. The fundamental problem is that hockey has the equivilent of runs, but the hits and walks that create those runs are missing from the statistical package.
Offence in hockey involves moving the puck through open ice towards the opponent’s goal. Finding open ice in the offensive end usually results in a scoring chance. Defense in hockey is about eliminating open ice and forcing the offense to give up the puck. The team that best controls the ice surface – by skating faster, by being disciplined, by filling the passing lanes and pressuring the puck, by winning the puck battles, by moving the puck – will get the most scoring chances.
In the NHL teams are close enough in quality to ensure that neither team can control the ice for the entire game. When control shifts, momentum appears to swing. Suddenly, the opponent is finding the open ice and forcing a chip and chase game on the formerly dominant team. Sometimes neither team can find any room. Back and forth it will go, but the team that carries most of the play usually wins. The top players – the ones who can do the most with open ice – lead the way. When a team can’t find open ice, their top players are often invisible.
Individual players do not – cannot – control the ice. It requires a team effort to deny ice to opponents, to cut off passing lanes and to pressure the puck. The entire team has to transition instantly. While a single player might occasionally go end to end, moving the puck down the ice is also usually a collective effort. This quintessentially team activity – finding or creating open ice with the puck, denying that open ice to the opponent without it – can’t be measured. We can see it – if questioned even a non-fan could identify which team is carrying the play – but we can’t count it.
In fact, it pollutes all of the individual numbers.
When Henrik Sedin is asked why he and his brother suddenly jumped to elite status at the age of 29, he says, “We’re the same players playing the same way we always have. The team is a lot better and so we are getting more chances and better chances.” If this is true – and I believe that it is – we have no way to tell how much credit to give Sedin for his numbers and how much credit belongs to the team. Ironically, one of the new statistics actually supports this hypothesis:
4. ZONE START
What is it?
The percentage of faceoffs players are on the ice for in the offensive zone. Neutral zone faceoffs are not counted.
What does it tell you?
How coaches use their different line combinations. Vancouver’s Sedin twins led the NHL in zone start last season, as they were on the ice for almost three times as many offensive zone draws as defensive zone ones.
Teammate Manny Malhotra, meanwhile, was in the opposite role, taking mostly defensive draws, which has a negative effect on plus-minus.
The most important thing this number tells us that the position of the puck when a player comes onto the ice has a very significant impact on his chance to score on the shift. This stat measures faceoffs, but the same dynamic must also be true for changes made on the fly. If Henrik comes onto the ice just as a teammate is moving up the ice with the puck, he is likely to do much better on that shift than if he steps onto the ice just as a Canuck turns it over.
The zone start stat reminds us that what happens on one shift has an impact on what happens in the next one. As the Canucks improved as a team from being good to being very good at controlling the ice, a significantly larger percentage of the Sedin shifts started with the puck in a favourable position for them. The result is that the same players are doing more scoring and have better underlying numbers.
Henrik Sedin’s numbers – traditional and new – do tell us that he is a very good player, but they can’t be more precise than that because we can’t tell whether he gets more help from his teammates than a star from another team gets from his. One of the reasons Jason Spezza’s scoring rate has dropped is because Ottawa as a team is no longer great at controlling the ice surface. A lot more of his shifts start with Spezza skating towards his own goal these days. That makes it a lot harder for him to score.
With lesser players – the real challenge in objective evaluation – the individual numbers mean even less because different players contribute in entirely different ways. There are reliable defensive players who seldom make a mistake, but provide almost no offence. There are other defensemen who can make a good pass and score once in a while, but make too many mistakes without the puck. There are forwards who can score but do little to help the team move the puck into scoring position. There are forwards who can win puck battles, kill penalties and even move the puck, but their hands are stone. Goaltenders don’t do anything except guard the net.
It is the mixture of all the skills, the collective skillset, that produces team strengths and team strengths win hockey games. Team speed. Team toughnesss. Team goaltending. Team defense. Team offense.
We can’t objectively sort out the individual contributions to those team strengths. Until we can find the hockey equivilent of singles, doubles and triples from the organized chaos of the game, the statistical evaluation of individual hockey players with disparate skills is a mug’s game.