Mike Bales took over for Giles Meloche as the goaltender coach for the Pittsburgh Penguins in the offseason after the 2012-2013 season. Everyone is familiar with the reasons why that happened, so I’ll skip to my question regarding the change. Did the coaching change it actually change Fleury’s performance?
The two pertinent questions are: did Fleury improve his ability to stop pucks from entering the net, and did he become more consistent in that skill? To examine this, I will use . This metric examines 5v5 shots and goals against, taking a broad measure of shot quality into account, to measure how goalie performed above average.
The data used for this analysis includes regular season and playoff games from the 2005-2006 season to the 2015-2016 season. I excluded games in which Fleury played less than 2 minutes of TOI in because of data quality issues (12 games, mostly from 2005-2006 and 2006-2007).
First, we will look at Fleury’s career, marking when the coaching change occurred.
Each dot is an individual game, while the blue line shows a smoothed line representing the general trend.
This graph treats each coach as a distinct era:
This perspective is too noisy to make any definite judgements from, though Fleury did not have any catastrophically bad games under Bales (less than -10 Adj. GSAA60).
The histograms below show the shape of the two coaching tenures. My reading is that Bales’ tenure is more to the right of zero (better), and more narrow (more consistent).
Fleury doesn’t look strikingly different during the Bales era, but maybe there is something we are missing.
When we take the mean (average) of Fleury’s Adj. GSAA60 in each coaching tenure, we see that Fleury did improve in terms of stopping pucks, increasing from .07 to .2 Adjusted Goals Saved Above Average Per 60.
The following graph shows where these two averages fall compared to other goalie’s seasons from 2005-2016. The purple line indicates Fleury’s play under Meloche, and the yellow line indicates Fleury’s play under Bales.
Based on this data, Mike Bales did improve Fleury noticeably.
The second question is whether Fleury became more consistent in his play.
My approach is to measure the standard deviation of the Adj. GSAA60 of each coach’s tenure. The higher the standard deviation, the less consistent the play.
Fleury did indeed become more consistent under Mike Bales. The standard deviation of his play using this metric dropped from 2.44 to 1.87.
The following graph shows where these numbers fall compared to other goalies. Again, the purple line indicates Fleury’s play under Meloche, and the yellow line indicates Fleury’s play under Bales.
However, looking at Fleury’s entire career, Fleury worked out most of the inconsistency in his game after his first 4 seasons (besides the 2010-2011 season). The increase in consistency under Bales does not look as significant in this light.
In summary, Mike Bales did increase Marc-Andre Fleury’s ability to make saves, and had a marginal positive effect on the consistency of Fleury’s play.
All of the R code used in this post is available on my , which also holds the data in CSV format for anyone who wants to look at the data outside of R.
PS: wrote up a more statically sound analysis of the data . I quote Gordon here:
We can pretty confidently stop worrying about any difference in mean performance level for Marc-Andre Fleury due to the change in Penguins goalie coaches. By contrast, Fleury’s consistency has clearly improved in the time Bales has held the job.
We ought to be careful yet about assigning to Bales a causal role in Fleury’s increased consistency, however. We have not ruled out alternative explanations. Perhaps all goalies have become more consistent in the past few years. Perhaps the underlying change really occurred at some other time, but we are picking up a difference at the 2013 offseason because that’s the only point in time we’re testing. There’s deeper we could dig into this question if we want.