Who are the best closers in baseball?

I’ve given up hope that the position of closer, at least as it’s currently practiced, will perish from the earth.  One inning, usually the ninth, with a lead of three or less.  The job is to come in and get three outs with the lead intact.  For today, I’ll set aside my usual objections this not being a good use of a good reliever.  I’ll just accept that this is the way that the baseball universe works.  It helps to dull the pain.
Well now, who are the best practitioners of the art of pitching one inning (and getting three outs) before the other team takes away your team’s lead?  It’s the guy with the most saves, right?  So, in 2006, that would have been Francisco Rodriguez (47 saves) and Trevor Hoffman (46).  Well, no, because suppose that K-Rod had 100  opportunities to do so (he didn’t), but only converted 47% of them.  He wouldn’t look like such a good closer then.  So, perhaps some sort of save opportunity conversion percentage makes more sense.  But then again, while we’re tinkering with saves (click on that link if you’re into the topic), let’s free ourselves from the nasty little restriction that it has to be the ninth inning.  Let’s find out which reliever is the best at protecting a small lead (< 4 runs) over one inning.
I looked for cases in 2006 which met the following criteria
1) The pitcher in the game is not the starter
2) He entered the game at the start of an inning
3) As he entered, his team was leading by between 1 and 3 runs
If he completed the inning (recorded all three outs) and the lead was still intact, he got credit for a “lead protect.”  If he surrendered the lead (the other team either tied the game or took the lead) within that inning, then he got a “lead surrendered.”  If he pitched more than one inning, I only counted the first one that he pitched.  This pretty much screws over anyone who came into the middle of the inning, and guys who usually only are brought in for one or two hitters (read: LOOGYs).  If the pitcher did not record all three outs but kept the lead intact, I simply discarded the case.  I wanted to make sure that I was measuring what a closer is usually asked to do.
To make it into these analyses, a pitcher had to have been in this situation at least 15 times.  Fifty-seven relievers qualified.  The top ten from 2006, with their number of saves in parentheses:

1) Mike Gonzalez, 100% (24)
2) Mariano Rivera, 96.2% (34)
3) Chris Ray, 96.0% (33)
4) Luis Vizcaino, 95.0% (0)
5) Francisco Rodriguez, 94.9% (47)
6) Juan Rincon, 94.7% (1)
7T) Joe Nathan, 94.4% (36)
7T) Salomon Torres, 94.4% (12)
9T) Joel Zumaya, 93.3% (1)
9T) Adam Wainwright, 93.3% (3)
The bottom five for the curious
57) Logan Kensing, 70.6% (1)
56) Armando Benitez, 71.4% (17)
55) Francisco Cordero, 71.4% (22)
54) Ryan Dempster, 71.9% (24)
53) Eddie Guardado, 72.2% (13)
Number 52, incidentally, was Danys Baez, to whom Orioles manager Mike Trembley turned after Chris Ray went down, despite the fact that Jamie Walker and Chad Bradford were both clearly having better seasons.  Baez protected 73.8% of leads handed him last year.  Sadly, his form this year may represent an improvement, but I don’t have 2007 numbers yet.  Trembley’s reasoning was that Baez had “closing experience.”  (To be fair, Baez was at 90.4 and 90.6% in 2004 and 2005). 
Despite the fact that it’s not always the best strategy to have your best guy out there in the ninth inning, most teams like to believe that they save the best for last.  Still, several teams got it wrong last year.  

  • Aaron Heilman (92.6%) of the Mets was better at this job description than Billy Wagner (88.1%), despite the fact that Wagner got 40 saves and Heilman got nothing. 
  • The normally statistically savvy A’s used Huston Street as their closer (82.3% protection rate, 37 saves), despite the fact that Kiki Calero (90.9%, 2) and Justin Duchscherer (89.5%, 9) were better at protecting leads. 
  • Dan Wheeler protected 92.3% of the leads given to him, while Brad Lidge protected 81.8% of his leads in 2006.  Perhaps the recently-fired Phil Garner’s decision to install Wheeler as the closer at the beginning of the season (after one bad garbage-time outing from Lidge) was not as rash as it seemed. 
  • Bob Howry, who protected 86.7% of his leads, must have been scratching his head that Cubs teammate Ryan Dempster (listed in the bottom five above!) was getting all the save opportunities.  On the North Side of Chicago, we all were too.
  • The Diamondbacks messed up big time when they traded Orlando Hernandez to the Mets for Jorge Julio.  The D-Backs installed 80% efficient Julio as their closer, despite the fact that there were three(!) other pitchers on the staff who were better at holding leads, Brandon Lyon (81.8%), Julio’s eventual replacement Jose Valverde (84.2%), and Vizcaino (95.0%).  To make matters worse, in the off-season, the D-Backs then traded Vizcaino to the Yankees for a 43-year-old starter with a bad back.

I wanted to see how consistent pitchers were over time, so I looked at this particular statistic from 2003-2006 and took an intra-class correlation.  The result was actually .06, meaning that there is very little consistency from year-to-year in performance in protecting leads.  Good results one year do not necessarily predict to good results the next year.  So, this statistic isn’t a good way to try to predict who would make a good closer next year.  It makes a little bit of sense.  Three to five batters is a very small sample size and a lot can happen in those times.  When those become your units of analysis, they’re going to be a lot more given to variability.
The way to fight against inconsistent year-to-year performance is to look at performance over a greater time frame, so I looked at pitcher’s total performance from 2004-2006 (3 years), with a minimum of 40 qualifying situations over the course of those three years.  The top ten over that time period were Gonzalez (100% still!), Guillermo Mota, Eric Gagne, Kyle Farnsworth, Salomon Torres (side note: just the Pirates luck, he’s 34 and excels in a very important skill, but no one recognizes that he does and so the Pirates can’t get anything for him), Valverde, Rivera, Vizcaino, Nathan, and Wheeler.  Of those, Torres, Vizcaino, Mota, and to some extent Farnsworth haven’t gotten a chance to prove themselves in the ninth inning.  The worst five were LaTroy Hawkins (73%), Octavio Dotel, Chris Reitsma, Shawn Chacon, and Ugueth Urbina.
When I dropped the inclusion criteria to 25 situations, a few interesting new names appear.  From 2004-2006, Boston swingman Julian Tavarez is actually 2nd with a 97% protection rate.  Blue Jays reliever Jason Frasor is ninth with a nifty 94% protection rate.  Frasor closed in 2004 for the Jays, and still pitches for them (quite well, actually), but has been sent to middle relief.  And, LaTroy Hawkins fans, it turns out that there is someone worse than LaTroy at preserving a lead.  Ray King only did his assigned duty 69% of the time between 2004-2006.
Saves do not make you a good closer.  Being a closer gets you a lot of saves.  If the job description of a closer is to pitch the ninth, get three outs and preserve a small lead and teams really buy into that (for good or for ill, and I’m on the side of ill) why do teams get it wrong so often?  Because closer is more akin to something to which you are knighted and annointed, rather than something that you… y’know… earn.


6 Responses to Who are the best closers in baseball?

  1. Sean Smith says:

    Pizza, how has Scot Shields done? His job is to get the 8th inning save, but lately hasn’t done a very good job of that.

  2. Pizza Cutter says:

    Shields was at 89% last year (K-Rod was better, but Shields outdid some “closers”). I’m getting this off Retrosheet data, so I don’t have 2007 data.

  3. tangotiger says:

    You can change your criteria to the following:
    – came into the game with the win expectancy greater than .500, and LI greater than 0.7
    – left the game with the win expectancy greater than .500
    – got at least 3 outs
    That satisfies all of your conditions, without forcing a “start the inning” restriction.

  4. tangotiger says:

    Thinking it some more… the “left the game” should be “left or ended the inning”. You’d probably need to do it on an inning-by-inning basis, and call it “inning protected”.

  5. John Beamer says:

    Leverage index is also a good metric to track closer usage. For instance in the 6th inning with a lead of 3 the LI is

  6. Pizza Cutter says:

    I thought about doing something a little more complex like using LI, but I wanted to keep the criteria as close to the actual role a closer is asked to play. Allowing guys who come in mid-inning introduces the confound of inherited runners, which closers rarely face. David Pinto did a really good job re-defining the save over at THT using a little more advanced method.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: