The final nail in the clutch hitting coffin?

A little more than half of one percent.  That’s it.
I can hardly think of another topic that sets the hearts of fans, analysts, and sportscasters more aflutter than the concept of clutch hitting ability.  Some players are just better when the game is on the line, right?  This one has been argued to death, with most of the statistical community finding that clutch ability is simply a myth and that players who always seem to come through in important situations also happen to be the ones that are really good hitters to begin with.  No one doubts that there are clutch situations (places in a game which are more important than others) or that clutch hits happen, but whether certain players have the ability to “turn it on when it counts” is dubious.  In fact, I intend to drive the final nail into the coffin of clutch.  (I realize that I won’t actually, but that just sounded cool when I wrote it.)
I can’t blame people for buying into the concept of clutch.  My training is as a psychologist, and I’m very familiar with all sorts of illusions that people believe in.  It’s not that people who believe in clutch are delusional or crazy, simply that they are poor processors of information.  That fact right there makes them more normal than anything.  It’s called the availability bias.  Most of the “clutch” hitters have a signature moment or two that you can immediately call to memory, plus a few other examples.  Why?  Not only because it’s probably been replayed over and over again, but it probably took place in a very emotionally laiden situation (like the playoffs or a late season game).  You can recall that, but not the 700 other plate appearances by that same hitter over the course of that season including the times he failed to deliver in an important situation.  Anecdotes are not science and the plural of anecdote is not data.
Here, I must acknowledge a very large amount of gratitude and perhaps plagiarism to Tango Tiger and his work on game leverage.  For those unfamiliar with the concept, leverage takes a look at how important each plate appearance is within the game in terms of how much it can swing the chances of who will win and lose.  Intuitively, you know that with two outs in the bottom of the ninth, two runners on, and the home team down by one that this is a very important moment in determining the outcome of the game (more so than if the visiting team were up by twelve).  The leverage index allows us to say how much more important a given game state is.
We start off by estimating the win probability of any given game state.  For example, let’s say it’s the seventh inning, with one out, no runners on, and the home team is down by one run.  Between 2000-2006, the visitors went on to win that game 67% of the time, while the home team won 33% of those games.  It’s just a matter of gathering the right data (Thank you, Retrosheet!) to compute the win probability for all possible combinations of inning (including whether it’s the bottom or top of the inning), score (more to the point, score differential), outs, and runners.  Now, at that point something will happen and at least one of those parameters will change.  The batter could make an out (number of outs) or he could get on base (runners) or he could hit a home run (score), but after that the chances of his team winning will have changed.  Maybe a lot, maybe not much, but they will have changed.  How much it changes is the key, and that is the foundation of the concept of leverage.  Leverage is a measure of how important a moment in the game is.  If the win probabilities might swing a great deal based on what happens in this at-bat (that is, “It all hinges on what Casey does here“), it has a high leverage rating. 
Tango’s article has all the gory details on how to calculate leverage, but let’s put it this way: a situation with a leverage rating of 2.0 is twice as important as the average situation.  So, the stakes, in terms of win probability are twice as important now than they are at an average point in the game.  These high leverage points are the points where clutch hitters should shine, if they really are out there.
In collaboration with David Appleman of FanGraphs (another fantastic site), Tom developed a measure of clutch rating combining using these two concepts of leverage and win probability.  I recommend reading up on the measure for yourself (he explains it better than I can), but the idea is that it rewards players for positive events in high leverage situations and punishes them for low-leverage situations (piling up stats in garbage time?)  Players who actually do perform better when the game is on the line, rather than in garbage time (i.e., those with clutch ability) would do well on this measure.  If a player puts up good numbers no matter the situation (that is, it’s not clutch ability, but his general ability that determines the outcome), his performance will be closer to zero.  Suffice it to say that I find this to be the best measure of clutch performance that I have ever laid eyes on.
If clutch ability exists as a skill, it should be somewhat consistent from year to year.  Previously, the standard method for measuring things like this has been to use year-to-year correlations, but I prefer to use intra-class correlation over several years worth of observations.  Year-to-year correlations are limited to two observations at a time.  Why use two only two years when you can use seven!
I took the Retrosheet PBP data for 2000-2006, and calculated win probabilities (and win probability added) and leverage indicies for all events in those years (about 1.3 million events… my computer groaned).  I only considered those events in which a batting event took place.  I did not count “gift” events, such as intentional walks or ROE, where the pitching/defensive team gave the player a free pass.  I calculated the new clutch index and got yearly totals for everyone in baseball who registered at least 100 PA in that year.  I normalized all totals by number of plate appearances and then calculated an intra-class correlation (using an AR1 covariance matrix, for you stat geeks) for the measure.
The intra-class correlation came out to be around .074, for a grand total R-squared value of .0056.  Intra-class correlation tells you how much of the variance in a measure, observed multiple times, is related to factors specific to an individual.  In other words, it tells you how much of the measure is related to talent and how much of it is related to other factors.
The contribution of talent to this clutch-ability measure?: A little more than half of one percent.  It was a significant number, but statistically significant doesn’t mean important.  Like a lot of other people, I’m finding that clutch ability does exist in the tiniest amount, and in such a tiny amount that it’s not worth looking for.  In fact, clutch hitting is actually 0.56% clutch ability and 99.44% other factors, I presume mostly general hitting talent and a little bit of luck.  What you perceive as a player being a good clutch hitter is simply his good hitting that’s been going on all along, but now you’re paying attention to it.  The ability to hit in the clutch is a lot like Bigfoot.  A lot of people claim to have seen it, but there’s very little in the way of proof of its existence.
With that said, I want props for going a whole post about clutch and never once mentioning David Ortiz.


2 Responses to The final nail in the clutch hitting coffin?

  1. Evan Brunell says:

    You mentioned Ortiz in the very last sentence. You lose!

  2. Derek Nelson says:

    For some reason, Evan’s comment brought to mind this story I heard once in my history class, about president Calvin Coolidge:
    Both his dry Yankee wit and his frugality with words became legendary. His wife, Grace Goodhue Coolidge, recounted that a young woman sitting next to Coolidge at a dinner party confided to him she had bet she could get at least three words of conversation from him. Without looking at her he quietly retorted, “You lose.”

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: