A Closer Look at Closers – Part One

Over the course of the next few weeks I will be primarily working with Closers – trying to determine the most effective ways to evaluate talent and quality at an inconsistent position that sure receives some hefty and consistent dollars.
This first part will introduce my opening step to a weighted formula to determine the value of a Closer, as well as discussing what a Closer is, and how we currently evaluate them.
Though this first part will focus solely on 2007, my study also consists of data from 2005 and 2006.
When compiling my data and examining game log after game log, I decided that my study and research should focus on some consistency, which can be hard to find for a Closer. 
I looked at the National League in 2005, 2006, and 2007, and wanted to limit my group to include only those who reached a certain criteria.  Initially I thought that anyone with 25+ saves in all three seasons should qualify.
Then, I actually saw the numbers and realized that would limit my study to include onlyJason Isringhausen, Trevor Hoffman, Billy Wagner, and Chad Cordero.
Suffice it to say, I wanted to have some more people in there.  With that in mind, I altered my criteria to simply those who actually were closers during those three seasons.  I also took into account the fact that some were demoted, promoted, or injured, and so my criteria called for 15+ saves in 2005, 2006, and 2007.
With those numbers, the nine Closers who find themselves under my statistical microscope are – Isringhausen, Hoffman, Wagner, Chad Cordero, Francisco Cordero, Brad Lidge, Jose Valverde, Brian Fuentes, and Ryan Dempster.
Yes, Francisco Cordero was in the AL for 2005 and some of 2006, however he has recorded 103 saves in the last three seasons and 60 of them were in the NL.  Plus, the whole idea of working with Closers stemmed from the idea that an inconsistent one-inning pitcher could receive a 4 yr/40 mil deal.
Simply stated, a Closer is a pitcher called on in the 8th or 9th innings, whose job is to seal the win for his team.  If he does his job he records a “Save.”  If the other team comes back to tie the game, he records a “Blown Save.” 
If you asked anyone about those stats before 1969, though, they would assume you were discussing hockey or soccer since saves are a relatively new statistic.
There are three ways a pitcher can record a save. I know this is a recap for many readers but it is important in the grand scheme of my study. The first way, which is how most people generally describe saves, involves the pitcher entering in either the 8th or 9th inning, with a lead of three or less, and preventing the other team from coming back to tie.
The second way is contingent upon when you enter the game and in what situation.  If you enter the game with the tying run on base, no matter the lead (usually extends it to a 4-run lead), and prevent the team from tying, you get a save.
The third way, which is how most middle relievers will rack up their 1-3 random saves per season, involves a pitcher going for the final three innings of the game – regardless of the score.  If the Phillies lead the Braves 9-1 and Ryan Madson pitches the 7th, 8th, and 9th, he gets a save.
If there are different types of save categories, doesn’t that mean there are different save types for each category?
Yes.  Plenty.  Think of it this way.  If you enter the 9th inning with only one out to go, and a 5-3 lead and bases empty, and you end the game, you get a save.  If you enter the 9th inning with only one out to go and the bases are loaded, and you end the game, you get a save.  One is clearly harder to do than the other and has a higher risk of resulting in a blown save, yet each ultimately results in the same statistic – a save.
With that in mind, I looked at the 9th inning and thought of all the possible situations that someone could receive a save.  In the 9th inning, there are 72 different ways to record a save, excluding what the pitcher does in the inning. 
If we count what the pitcher does, either giving up a run with a two-run lead or two runs with a three-run lead, and so forth, in the 9th inning there are 144 total ways to record a save.  I will get more into these different ways in Part Two, however the basic idea is that there are eight situations of baserunners (empty, 1st, 2nd, 3rd, 1st and 2nd, 1st and 3rd, 2nd and 3rd, bases full) and 18 different variations of these eight situations.  These variations include entering with 1 out, with 2 outs, with no outs, entering with 1-run, 2-run, or 3-run leads, and more of the same.
144 different ways can a pitcher record a save in the 9th inning, depending on how many outs he records, what the baserunning situation is, and how many runs he gives up.  Yes, this can be said for many other statistics, but Saves generally only span 2-innings MAX, and so the huge number of different types means a bit more here.
I am not dealing with “clutch” in my study.  To read some fascinating insights into relief pitching and relief clutch, read Pizza Cutter’s articles on the subject.
Instead, I am looking at what actually happens and how it happens, not the potential of why it happens.
Many people will look at two, and only two, stats when determining the quality of a closer – total saves, and percentage of successful saves (saves/save opportunities).  It has been pounded into our heads as a barometer and these statistics are supposed to inform us that the “best” closers are the ones with either the most saves or least blown saves.
What I am contending is that if there are 144 different types of 9th inning saves, and the barometer is the sum of all converted opportunities, regardless of the type of save, the needs of your team, and the situation at hand, it is impossible to equate total saves to quality.
Think of it this way – Closer A and Closer B both have 30 saves.  Closer A has 6 blown saves while B has 4 blown saves.  With those numbers, which are usually the only ones readily available, we assume that Closer B was better.  After all, he blew less saves.  What if the 4 saves he blew were all 3-run leads with bases empty and only 1 out to go in the 9th inning, though, which is the least dangerous save situation of the whole 144.  And what if the 6 blown saves of Closer A were all games he came in with runners on third base and no outs, or games where he entered in the 8th inning.
It becomes very difficult to gauge the “better” factor with just those numbers.
Regardless, even looking at hypotheticals like that, which take into account different types of saves, we cannot determine true quality because it does not take into account the needs of the teams these closers are on – which is ultimately the point of the closer.
When we discuss who the best closer is, what are we asking?  Are we wondering who was best with the most pressure?  Who posted the best numbers?  And if we are talking about numbers, what numbers are the best numbers?
These questions, and more, can cause a headache.  My point here is that we cannot compare closers to each other or determine true quality and effectiveness without analyzing what each closer did for his team.
In order to do this we need to find the number of games that each team won in a save situation (meaning no walk-off wins or 3-inning saves) and add it to the number of Blown Save-Losses because that tells us the true number of save opportunities each team had.  I call that a TSO – Team Save Opportunity.
Jose Valverde had 47 saves this year, leading the NL, however the DBacks had 64 Team Save Opportunities, whereas Ryan Dempster’s Cubs only had 48 of those games – sixteen less than the DBacks.
Dempster had 28 saves, much less than Valverde, but his conversion rate (28/31) was higher.  Valverde had more saves, but he also had more opportunities because his team played a different way and, as a team, played more close games that needed saving.  And even though Dempster’s percentage was higher, he also had less opportunities to blow saves.  If he had the 54 appearances of Valverde, he may have also blown more saves and had a worse conversion rate.
What we need to do here is level the field of play between those on teams with many save opportunities and teams with fewer.  After all, it is not Dempster’s fault that the Cubs had a better offense and blew teams out more than the DBacks.  He was not needed as often as Valverde and so his raw save and blown save totals do nothing but compare one number of Dempster’s to the overall need of Valverde and the needs of the Diamondbacks.
To really do this, the effectiveness of one pitcher to his team needs to be compared to the effectiveness of another pitcher to another team.
The DBacks had 64 TSO’s and Valverde had 54 opportunities.  This means that Valverde appeared in 54 of the 64 total save opportunities for his team, or 84.4 %.
That 84.4 % tells us he was durable since the team had so many potential save opportunities and his appearances were so high.
The Cubs only had 48 team save opportunities and Dempster only had 31 attempts.  His appearance rate would be 31 of 48, or 64.6 %.
Yes, Dempster was hurt, but this does make sense because you cannot be more effective (positive or negative) for your team if you are not involved as often as possible.  The fact that other pitchers were involved in over 1/3 of the Cubs save opportunities says that Dempster was not truly effective in making appearances.
To see the order of the nine closers in terms of Appearance Rate, look at the table below. The table shows the saves and save opportunities of the individual, as well as the total real save opportunities of the team, and then the Appearance Rate.

F. Cordero 44 51 58 87.9
Valverde 47 54 64 84.4
Hoffman 42 49 60 81.7
Wagner 34 39 48 81.3
C. Cordero 37 46 59 78.0
Isringhausen 32 34 46 73.9
Dempster 28 31 48 64.6
Lidge 19 27 55 49.1
Fuentes 20 27 59 45.8

Despite this stat being useful to tell us how durable or useful a closer can be in making appearances based on team need, it does not tell us how successful they were in actually converting these saves. Just because Valverde appeared in 54 of 64 team save opportunities for the DBacks does not mean he converted 54 saves – just that he made 54 appearances.
After careful thought, I came up with “Save Rate”, which takes the total number of saves by a closer and divides it by the total number of team opportunities.  This statistic takes the Appearance Rate to the next level.  Since closers can have a high Appearance Rate but low number of saves or low save percentage, Save Rate balances that out.
Save Rate lets us know how successful a Closer was in recording saves relative to the percentage of his team’s save opportunities.  It tells us how successful one was based on how effective he was in fulfilling his team’s need.
Essentially, it rewards those with more saves in less team opportunities, and takes away from those with less saves in more opportunities.
Valverde had 47 saves out of 54 chances, and his team had 64 real save opportunities.  His Save% would be 47/54 and his Appearance Rate would be 54/64.
His Save Rate would be 47 (# of saves)/64 (# of total team chances for a save), which comes out to 73.4 %, meaning that Valverde successfully saved 73.4 % of the DBacks team save opportunities.
Francisco Cordero of the Brewers was 2nd in the NL with 44 total saves.  He also blew seven saves giving him 51 opportunities.  His Save% was 44/51, very similar to Valverde, but his Appearance Rate was higher because the Brewers had six less team save opportunities and he only had three less appearances than Valverde.
His Appearance Rate was 51/58, or 87.9 %.  He appeared in more games proportionate to his team’s need.
His Save Rate would be 44/58, or 75.9 %, higher than Valverde’s.
To see the nine closers in order of Save Rate, look at the table below.  Again, it lists the total saves and opportunities of the individual, as well as the total team opportunities, and then the actual Save Rate.

F. Cordero 44 51 58 75.9 %
Valverde 47 54 64 73.4 %
Wagner 34 39 48 70.8 %
Hoffman 42 49 60 70.0 %
Isringhausen 32 34 46 69.6 %
C. Cordero 37 46 59 62.7 %
Dempster 28 31 48 58.3 %
Lidge 19 27 55 34.5 %
Fuentes 20 27 59 33.9 %

It makes sense that Cordero would be higher because even though his save totals and appearance totals were slightly less, he was involved in a higher percentage of his team’s chances and he converted successful saves at almost an identical number and percent.  Basically, he had less opportunities and still did the same exact thing – not the same ratio, but the same thing.
This does not necessarily mean Cordero had a better season.  This is merely one part of a two or three part article series and Save Rate is only the first part to a weighted system that should be able to determine who the best Closers are based on statistics that essentially define a good Closer.
Next week I will get into the different types of saves featured in the data sheets and discuss their importance in determining quality and effectiveness. WPA and Win Predictors will be discussed as well.
I will also look at raw numbers to help come up with the Seidman Closer Model to properly evaluate Closers.
In closing (pun very intended), I just want to add that the Closer position has become such a fickle one over the years that these evaluations need to be done on a year to year basis.  Jose Valverde was arguably one of the best NL Closers in 2007, and somewhat of a replacement, or makeshift closer, in 2005.  Brian Fuentes was dynamite in 2005 and still pretty good in 2006, yet so bad in 2007 that he lost his job.
It is remarkable how inconsistent Closers are, and that is one of the primary reasons (along with playoff success) why Mariano Rivera will go down as the greatest ever.
Lastly, of the nine closers used in this ongoing study:

  • Lidge and Fuentes were demoted in 2007
  • Lidge and Valverde were traded to new teams
  • Dempster is likely going back to the starting rotation
  • Francisco signed a huge four-year deal with a new team
  • Billy Wagner changed teams from 2005 to 2006

The only NL Closers that have actually kept their job for the same team between 2005 and 2007 are – Trevor Hoffman, Jason Isringhausen, and Chad Cordero.


6 Responses to A Closer Look at Closers – Part One

  1. Pizza Cutter says:

    One minor tweak. When you look at team opportunities, be sure to take out ninth-inning comebacks by a team at home. For example, the Cubs are playing at Wrigley, and are down 7-5 in the top of the ninth. Dempster may pitch the ninth (given Dempster’s track record, that’s a scary thought), but it’s certainly not a save situation for him. Then, in the bottom of the ninth, the Cubs come up with three runs and win 8-7. It’s a one-run win for the Cubbies, but not one where Dempster (or any other Cub) had a chance to record a save.
    I still maintain that the best “closers” in baseball last year were Hideki Okajima and Tony Pena, who had something like 6 saves between the two of them.

  2. Thanks for the heads up. I’m making the slight changes as I type.
    Okajima and Pena may have potentially been the best relief pitchers suited to close based on their stats and abilities, but they were not “Closers” which is essentially one of my biggest points.
    Closer has become a premium position and does not take into account those who may be best-suited to record saves, but rather those handed the title of “Closer.”

  3. Just so anyone who reads this now knows, I updated the article and data sheet to remove any walk-off win since that does not count as a save opportunity for the winning team, though it can count as a blown save for the losing team.
    The numbers were virtually the same except for Billy Wagner, who moved ahead in some areas.

  4. Hey everyone… sorry for closing the discussion board yesterday. I was thinking of halting it until the final article in this series was published, but I changed my mind.
    Before I left, loyal reader and commenter “dan” wrote about how he felt WPA would be a better estimate for what I am looking for.
    While I plan to discuss WPA and its merits in the next article on this subject, I will say that WPA is definitely better when differentiating between a 1.1 inning save and a 0.1 inning save, but it does not take into account something I am stressing here, which is that some closers are on bad teams that win less games and some closers are on teams that have many more/many less save opportunities.

  5. dan says:

    Thanks for clearing that up…. I was all ready to invoke my first amendment rights. I’ll wait ’till the next part for further comment(s)

  6. Haha, no problem Dan. If you want to look at WPA in relation to evaluating Closers, a way to do so would be to compare the Wins Created to the Total Number of Team Wins. Let’s talk about this next week when I actually discuss WPA and Win Predictors in the evaluation part two.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: