An Inside Look at Analytics With Eric Tulsky

As Vice President of Hockey Management and Strategy for the Carolina Hurricanes,
Eric Tulsky
provides and analyzes data to assist the hockey operations department and coaching staff.
That's about the most unambiguous way to describe the role of Tulsky, who pioneered the Canes' ever-growing data department in a full-time capacity more than five years ago. Tulsky is
a smart man
with a
rich scientific background
, and while the world of hockey analytics is a secretive business - it kind of has to be in order for teams to gain even the slightest bit of competitive edge in the salary cap era - Tulsky provided an inside glimpse at his role during an Ask Me Anything (AMA) session on
Reddit
.
Here are the highlights from
his AMA
, the latest in a weeklong series as a part of Fan Appreciation Week.

Editor's note: Some questions and answers have been condensed or edited for grammar or clarity.
How likely are you to score an empty-net goal, and how much does it increase over time? What is the actual best time to pull a goalie?
I remember having a conversation with someone once, a casual fan who was questioning the value of pulling the goalie. "It seems like it never works," he said. "That's true," I replied, "the only thing that works less often is not pulling the goalie."
The best time to pull a goalie is a complicated question. The arithmetic is pretty straightforward if you assume a certain rate of scoring or giving up a goal, but things are never that simple in real life.
Published estimates say that the best time to pull the goalie when you are down by two is with seven minutes left, but those estimates are based on looking at scoring rates in the existing environment. Right now, teams play their best offensive players for the vast majority of their 6-on-5 time, but you can't just plan on having your top line play for seven minutes straight. Does their ability to score drop as they get fatigued? Is it risky to try to change on the fly with no goalie in net? How much lower is the second unit's scoring rate? If you pull that early, are you just rotating the two units for seven minutes, or do you have to give your six least-dangerous scorers a shift at some point?
Like many things, something that seems like a simple question at first glance has a lot of ways the analysis can go wrong.
If they added a "2 goal" shot line, statically where would it be?
I'm generally more open to experimenting with rules than a lot of people are, but this is one that I'd have a hard time with. Not because I hate the idea, but because I'm not sure it would impact play the way you'd like it to.
The idea of making a rule like that would presumably be to open up the interior. You create high-value shots on the perimeter, and then the defense has to stretch a little more, and it creates more room for skill plays. That's more or less what happens in basketball.
But there's a key difference in hockey: You can defend the perimeter without stretching out to pressure the puck directly because you can block a shot from 30' away. If anything, this might actually make teams
sag more
because it increases the penalty if you pressure the top of the zone and get beaten by a pass.
I guess I haven't directly answered your question, but unless you put the line far enough away that teams would largely ignore it, I think it might have the opposite effect of what people are imagining. So my instinct is not to have a line like that - all goals should count as one, except maybe lacrosse goals.
Is plus/minus a good stat?
The traditional way that plus/minus is formulated is kind of silly; lumping in things like empty net goals and shorthanded goals really screws things up. There are examples of seasons where two teammates had similar results at even strength but saw empty net and shorthanded goals produce a 30-goal swing, which can obviously get in the way of a lot of attempts to use traditional plus/minus to draw a conclusion.
But if you narrow things down to just even strength, or just five-on-five, there's nothing wrong with it. Like any stat, you have to be careful about how you use it - it can be influenced by who the player plays with and against, and the sample sizes are small enough to produce some fluctuations that don't mean much, but with the right context and understanding, obviously it's valuable to know if a player can repeatedly help his team outscore his opponents.
What does a typical trade negotiation look like?
Typical? The median is probably something like this:
"Would you move [insert name of good player]"
"No chance."
Can you talk about the difference between the data available to NHL teams and publicly available data? How useful are the models that use publicly available data, and do you ever look at them?
The league publishes a lot of information about each game: what time each shift started and ended and a variety of events (shots, hits, turnovers, etc) that took place during that shift. There is a lot that you can do with that - remember, using that information to answer questions is how I got my start in working with hockey data.
But there are gaps in the data as well; it can tell you that someone took a shot from the faceoff dot at 8:32, but not much about how that shot came about or what other options he might have had. So the public models generally rely on inferring a player's impact from looking at how the team's results change when he is on the ice - and that can be done very well, and can be powerful. But when you want to move from talking about what happened to talking about why it happened or what could have been done differently, you often need a richer data set.
And yes, we absolutely do look at work done in the public sphere. A couple of years ago,

about it at the Sloan Sports Analytics Conference. One of the things I talked about there was how external work can help counteract a bias that I don't hear discussed very often.
No two people look at things exactly the same way; everyone has traits they value a little more than average or a little less than average. When a player is available - whether it's through the draft, the trade market, or free agency - more often than not, he will end up being acquired by someone who values something about his play more than most teams do.
So over time, you end up with a roster full of players who you like more than most teams do, and so you think your team is better than most people do. That's not just a data thing; it's true regardless of how you form your opinions about players. And external opinions can be helpful for counteracting that bias; it can highlight for you where those biggest disagreements are and give you a chance to re-examine your perspective and think about what you might be missing.
(If you are allowed to share) ... How does the Hurricanes' analytics department distinguish itself from other teams' departments? Broadly speaking, what analyses or evaluations does this team have that give it an advantage over the rest of the NHL?
It's a pretty secretive business, and it's hard to know what other teams have -- if you can get them to tell you, let me know and we'll compare notes. We could be dead last in innovation and I don't know if I'd have any way of knowing that.
So it's hard for me to answer the question about comparing ourselves to other teams directly, but I will say that we've gotten a lot of mileage out of a focus on communication and helping to organize and present information of all types.
The Canes' drafting strategy appears to be looking for high-upside skill players at every stage in the draft and avoiding "safe" low ceiling picks. Is this because it is easier to find value in veteran 3rd and 4th line guys via trade or free agency?
I don't think there are any safe picks in the NHL Draft.
In the NFL, a team will be looking at a 23-year-old who's going to play for them next year; the scout's job is mostly about projecting how his skills will translate. But in the NHL, we're drafting an 18-year-old - in addition to estimating how his skills will translate, our amateur scouting group has the daunting task of projecting what he will look like after five years of development.
Even the top pick in the draft isn't safe - it might be a reasonably safe bet that he'll play in the NHL, but he has a long way to go to become the player you need him to be. There's a huge range of possible outcomes for every player in the draft, and the goal isn't to pick a player with a high floor or a high ceiling; it's to think about all of a player's possible outcomes and pick someone who would be a good bet.
How does your PhD/past career help in your role? What was the transition to sports data science like?
Honestly, the technical side of my background is almost completely irrelevant. If we ever want to deposit a nanometer-thick layer of insulating material on the outside of our equipment (or our players, I suppose), I will definitely be heavily involved in that, but otherwise, not so much.
But there are a lot of soft skills that translate. The thing that makes chemistry difficult - especially the branch of chemistry I worked in - is that it often lacks clear and unequivocal evidence about what is happening. You can make a variety of measurements that might suggest that the molecules did a certain thing, but those measurements rarely look exactly how you expected and often disagree with each other to some degree. So being successful in that field requires being good at keeping a bunch of possibilities in mind - you have to have good judgment not just about what the most likely explanation is, but also about what other explanations are possible and what their implications would be and how you can test them. The exact same thing happens with sports data.
And I've also leaned quite a bit on skills I picked up in my time in industry. Working at a global conglomerate for five years gave me a lot of exposure to industrial processes; I got a lot of training on things like group decision-making and strategic planning and employee mentoring. And then working at start-ups for the next five gave me a chance to see which elements translate well into an environment that has pace and pressure more like the NHL.
Hi Eric, how frequent are your interactions with Darren Yorke on the non-analytics side of the house?
"We are constantly collaborating with one another. We're not an organization that is built on the poles. There isn't an eye test vs. analytics. We're all on the same team, trying to gather as much information as we can to make the best decision possible. -
Darren Yorke
"
- Eric Tulsky
There's much more to take in with Tulsky's full AMA, in which he discusses more about analytics, his role with the team and his favorite dessert.
Fan Appreciation Week AMA schedule | Join the Canes discussion on Reddit