Can we classify every pitch?
December 22, 2007 8 Comments
What if we knew what type of pitches every major league pitcher threw? What if we had detailed pitch-by-pitch data about how he used those pitches in every game situation? What if this information was accurate and freely accessible to baseball researchers?
Let’s begin with some history. Since Sportvision’s PITCHf/x system was unveiled during the 2006 playoffs, people have been thinking about using the detailed pitch data to classify pitches by type. Reference this comment by MLBAM’s Director of Stats, Cory Schwartz:
“When the system is installed in all 30 ballparks, it will provide unprecedented accuracy, consistency and depth of data to the measurement of speed and trajectory of each pitch,” Schwartz said. “Ultimately we’ll be able to use this data to determine the pitch type in real time and with greater accuracy than ever before. By recording all of this data in real time, we can provide it to broadcasters such as FOX, in-stadium scoreboards, fans via Enhanced Gameday, clubs and other business partners.
It wasn’t long before Baseball Analysts’ Joe Sheehan was leading the public research down that path, too, publishing articles in the spring of 2007 about pitch classification for pitchers like Jeff Weaver, Mike Mussina, and Kenny Rogers, using the data from the 2006 playoffs.
In April 2007, the PITCHf/x system was installed in nine ballparks, and this produced a wealth of data that encouraged more people to join the analysis fun. Dan Fox, Bill Ferris, and Steve West were among the leading PITCHf/x researchers in the first half of 2007, and although the work in the field covered a number of topics, pitch classification was often at the forefront.
Soon the quest turned toward developing a set of rules to classify pitches for many pitchers, perhaps for every major league pitcher. John Walsh published the early definitive article on this topic. In August, the analysis really began to heat up; for example, see these articles from John Beamer and Joe Sheehan. The quest for a pitch classification algorithm was on.
Several people mentioned clustering algorithms as a potential tool for universal pitch classification. K-means clustering was examined, but it requires the user to specify the number of clusters in advance, which limits its usefulness over the whole major league population, where pitchers have widely varying repertoires. Josh Kalk and Joe Sheehan made the most headway with pitch classification by clustering algorithm. Josh Kalk graciously made his results freely available in the former of player cards for every major league pitcher with at least 100 pitches recorded by the system.
Unfortunately, I’m not enough of an expert in clustering algorithms to replicate their work, and neither Joe nor Josh has made their algorithms public. How do we evaluate the effectiveness of their algorithms, or even attempt to construct a different classification method on our own?
I should mention that there is another approach to pitch classification that has been applied quite successfully by John Walsh in answering a number of questions, not least in his article in The Hardball Times Baseball Annual 2008 investigating the cause of platoon splits. John classified pitches into boxes based on speed and horizontal and vertical spin deflection (pfx_z vs. pfx_x, termed “break” by many analysts). This approach appears to work well for large groups but less well for individual pitchers whose fastball speed or pitch movement varies from the norm.
I began investigating the repertoires of individual pitchers with the hope of developing a set of rules I could apply generally to all pitchers. It soon became clear to me that this was not going to be an easy task. Let me recount some of the steps I took, how my results compared to those from Josh Kalk’s algorithm or John Walsh’s approach, and what I learned along the way.
For my first real pitch classification challenge, I chose Boston Red Sox closer Jonathan Papelbon. Try using pitch speed and horizontal and vertical spin deflection to identify those pitches. I don’t think you can do it. Even Josh Kalk’s clustering algorithm fails pretty badly for Papelbon. It’s clear that he throws a fastball, but what type of fastball, and does he throw more than one type? What are his off-speed pitches?
To answer these questions I turned to a paper by Physics of Baseball professor Alan Nathan on calculating spin parameters from PITCHf/x data. I developed a classification technique based on solving some approximate equations Alan presented for spin direction and spin rate. Spin direction and spin rate parameters contain the same information as the vertical and horizontal spin deflection parameters, but they present the information through the lens of a different coordinate system, and sometimes that can be very helpful in differentiating pitches.
In Papelbon’s case, taking the spin view rather than the “break” view is almost essential. Spin direction clearly delineates his curveball and slider, but the difference in their spin rates results in similar overall spin-induced movement that makes them difficult to tell apart on the “break” graph. The four-seam and two-seam fastballs have a similar situation–easily differentiated by pitch speed and spin direction, but the lower spin rate of the two-seamer brings its “break” back close enough to that of the four-seamer that they are hard to tell apart. And finally, while an off-speed pitch, perhaps a changeup, is somewhat identifiable in the speed-break graphs and probably correctly identified by the clustering algorithms, it’s even easier to identify in the speed-spin direction graph. To find its name–the split finger fastball–I turned to scouting reports, which seemed to fit the data. I also found that Papelbon used his splitter mostly against lefties and his breaking ball mostly against righties.
Papelbon turned out to be the classic case where speed-break graphs and clustering algorithms based upon them spit out wrong answers, but spin parameters make his pitches easy to classify. What I had thought was a hard case turned out to be an easy case when the right tools were applied. Other pitchers would not be so easy, and I would find that spin parameters, while continuing to be very illuminating, unfortunately did not lend themselves to a consistent set of rules across all pitchers. Nonetheless, what tentative classification rules did I develop from the Papelbon analysis?
Speed vs. spin direction can be used to identify pitch type clusters that do not appear distinctly in speed vs. break.
- For right-handed pitchers, four-seam fastballs generally have higher speed and a spin axis at a lower angle than two-seam fastballs.
- For right-handed pitchers, curveballs generally have a lower speed and a spin axis at a lower angle than sliders.
- For right-handed pitchers, split-finger fastballs have a lower speed and a spin axis at a higher angle than other fastballs.
- Split-finger fastballs are used primarily against opposite-handed hitters.
Next, I applied the first rule to separate Edinson Volquez‘s four-seam and two-seam fastballs, again something that clustering algorithms based on break have trouble doing (Kalk, West).
For my next challenge, I took on Greg Maddux. Maddux’s pitch repertoire is a tough nut to crack by any method, since by some accounts he purposefully varies the speed and movement on his pitches by slight amounts all the time. The main addition garnered from the Maddux analysis was a first encounter with the cut fastball as a distinct pitch. In retrospect there may have been a few cutters hiding out in the fastball groupings for Papelbon, but they weren’t many in number and they were similar enough in movement to the four-seam fastball that I didn’t detect them as a separate group. With Maddux there was obviously a separate fastball group with slightly lower speed and a spin axis at a lower angle. The speed and spin axis were too low to be regular fastballs and too high to be sliders. In addition, he used the cutter primarily against lefties, whereas sliders and regular fastballs were used more against righties. So we add a couple more classification rules to our list.
- For right-handed pitchers, cut fastballs have a speed roughly 2-4 mph slower and a spin axis at a somewhat lower angle than a regular fastball.
- Cut fastballs are used primarily against opposite-handed hitters.
I then tackled underhand pitcher Chad Bradford, but submariners and true side-armers are such a different breed that I’m going to leave them out of the discussion for now and assume they must be classified by a separate algorithm. Fortunately there are maybe only a dozen of them in the majors, so dealing with them separately or simply excluding them from some studies is not unreasonable.
My analysis of Zack Greinke is less interesting for finding new rules, although it did confirm that speed and spin direction were more useful in separating his slider and curveball than speed and break. However, something at least suggested by Greinke’s case is that a pitcher’s repertoire may look different in relief than in the starting rotation. He may throw harder in the bullpen than in the rotation. In Greinke’s case, he only threw 1-2 mph harder in his relief role, but I suspect the difference may be more for some other pitchers. In addition, a reliever might restrict his repertoire to rely primarily on one or two of his best pitches, while a starter may find more need for third, fourth, or even fifth pitches. This is an avenue for further investigation, but I will list it as a tentative rule.
- It may be useful to analyze a pitcher’s relief outings separate from his outings as a starting pitcher.
Next up is Eric Gagne, for whom the speed/break clustering is actually pretty straightforward. For some reason, Josh Kalk’s player card for Gagne gets some strange results, totally missing his slider. That may be a problem with some of the corrections Josh applies to his data (park, air density), or it may be a problem with his clustering algorithm. I noticed a few other things in the Gagne analysis that are worth a comment. First, many of Gagne’s sliders had a very low calculated spin rate, between 200-700 rpm, compared to 2000+ rpm for a typical fastball. This effect occurs because most of the spin is around the direction of travel. Only the component of spin in the x-z plane affects the break of the pitch, and that is what can be calculated by my analysis. The low measured spin rate is a signature of the slider, and I’ll list that as a new rule here, but we’ll add a qualifier later on.
Also, scouting reports will tell you that Gagne’s off-speed pitch is a “Vulcan” changeup, describing the grip he uses for that pitch. It’s a hybrid of a split-finger fastball and a normal changeup, and it moves like a hybrid pitch. It has the speed of a typical changeup, but the angle of its spin axis is quite a bit greater than that of Gagne’s fastball. Tweener pitches like this are what make comprehensive rule construction difficult. For Gagne we define it as a changeup since its speed is 10 mph slower than his fastball, but will that be universally helpful in distinguishing splitters from changeups? Possibly. Papelbon is a counterexample; however, where his off-speed pitch is about 10 mph slower than his fastball and with a similar spin axis offset as Gagne’s Vulcan change. In terms of absolute speed, Papelbon’s splitter runs 84-91 mph and Gagne’s changeup 80-87 mph. Hmm. Both these rules need to be refined, but we’ll list them anyway.
- Pitches with very low calculated spin rates tend to be sliders since much of the actual rotation is around the direction of travel (and is thus not measured).
- Split-finger fastballs tend to be thrown harder than changeups.
With Josh Beckett, I encountered another important principle and possibly a difficult hurdle for a universal classification system. The speed of Beckett’s fastball, as recorded by the PITCHf/x system, varied by as much as 5-6 mph between starts. It’s possible that was a real effect, or perhaps a data error, but in any case, it made his fastball difficult to distinguish from his changeup. In order to distinguish the pitches, I identified by speed which pitches were fastballs on a start-by-start basis and then normalized the median fastball speed in any given start to Beckett’s overall average fastball speed. Josh Kalk’s park correction seems to be able to overcome this problem in Beckett’s case.
I also observed with Beckett that his four-seam fastball, two-seam fastball, and cut fastball were difficult to distinguish. Two-seamers and cutters have lower spin rates than four-seamers (and with most pitchers, slower speeds, but not for Beckett), but the transition between types was smooth rather than clustered. I didn’t come up with a good solution for this problem, finally just drawing a couple lines through the data where it seemed best to me (where I found minima in the spin direction histogram). I did note that Beckett used his two-seamer more often to right-handed hitters than to lefties, something I also observed with Greg Maddux, but it’s a tendency rather than a hard and fast rule.
- If a pitcher’s fastball speed varies greatly from appearance to appearance, pitches may need to be classified on an appearance basis rather than a season basis.
- Two-seam fastballs tend to be used more often against same-handed hitters.
- Cut fastballs tend to have slower spin rates than four-seam fastballs.
- Two-seam fastballs tend to have slower spin rates than four-seam fastballs.
The next pitcher under my microscope was Joba Chamberlain. His four pitches–fastball, slider, curveball, changeup–were readily identifiable, either by speed/break or speed/spin analysis. (For some reason, Josh Kalk’s algorithm inexplicably identifies all his off-speed pitches as sliders.) Chamberlain didn’t break any of my previously established rules, so while very interesting from a baseball perspective, he’s fairly uninteresting for algorithm development.
Finally we arrive at the start of my tenure with MVN, where I began with an article on John Smoltz. As I did with Josh Beckett, I found it useful to classify pitches for Smoltz on a start-by-start basis because of significant variations by start in spin/break as well as speed. This approach was helpful in distinguishing fastballs from sliders and changeups, or at least that’s how I used it on the first pass. A closer look at the data, combined with reading of scouting reports, informed me that Smoltz had a split-finger fastball as well as a changeup. Apparently at one point in his career the splitter was a primary out pitch for Smoltz, but it seems to have fallen into disfavor in the last couple years because of injuries. Nonetheless, a careful attention to the game-by-game splits of spin direction and speed indicated the presence of a split-finger pitch thrown with slightly greater speed and a greater spin axis angle than the changeup. An exact dividing line between the two was difficult to establish. Given what I learned; however, I can refine my earlier rule from Papelbon and Gagne that simply stated that splitters were thrown harder than changeups.
- Split-finger fastballs tend to be thrown a few mph harder than changeups.
- Split-finger fastballs tend to be thrown with a spin axis at a somewhat higher angle than a changeup.
- Split-finger fastballs tend to be thrown almost exclusively to opposite-handed hitters.
- Changeups also tend to be thrown to opposite-handed hitters, but the tendency is much weaker than for splitters.
- If a pitcher’s speed, spin, and/or break vary considerably from appearance to appearance, pitches may need to be classified on an appearance basis rather than a season basis.
After I published the article on Smoltz, I got some good feedback at the Baseball Think Factory newsblog. It’s apparent to me from that discussion that I’ve lumped the cut fastballs and sliders together in my Smoltz analysis. I can see a handful of 90+ mph pitches which are clearly cutters, but as of yet I have not been able to determine a way to reliably distinguish the two pitches for John Smoltz.
My next article was on Mariano Rivera, who supposedly has a very simple repertoire–one pitch: the cut fastball. In fact, I found that he threw two pitches, mostly the cutter but also a regular fastball, subsequently confirmed by searches of the scouting reports. This fact is visible but overlookable in the speed/break graphs, and in fact Josh Kalk’s clustering algorithm only sees the cutter. Looking at spin direction, however, two clusters are readily identifiable, and this identification is further confirmed when one notes that Rivera only throws the regular fastball to righties. As an aside, I went the extra mile on identifying the pitches at the cutter/regular fastball boundary by examining the spin direction on an appearance-by-appearance basis. This is probably not strictly necessary for a universal algorithm that is allowed to misdiagnose a pitch here or there, but for getting closer to 100% accuracy it was useful in this case.
My most recent pitcher analysis was for Joakim Soria. Finally we find another pitcher for whom Josh Kalk’s algorithm and I are in pretty good agreement. One area of difference is in what to call his fastball. It has a fair amount of cutting movement, and his catcher called it a cutter, so I called it a cutter, but I wouldn’t argue too strenuously with someone who wanted to call it a regular four-seamer. It’s on the borderline, and one facet of designing any universal algorithm is going to be a decision on what to call pitches like Soria’s fastball. One interesting thing to note for Soria is that his slider looks like a curveball from many other pitchers, in its speed, break, and spin direction. What identifies it as a slider for Soria is comparison to his curveball, which is slower, breaks more, and has a spin axis at a smaller angle. Soria brings up more questions than new rules, but we’ll list them anyway…
- Should fastballs without much lateral movement that are used as a pitcher’s primary fastball be labeled as cut fastballs or four-seam fastballs?
- Should sliders and curveballs be identified primarily by comparison within the pitcher’s own repertoire or with respect to the sliders and curveballs of the rest of the league?
With Soria I also observed that he throws his slider almost exclusively to righties and his changeup almost exclusively to lefties. Not every pitcher is nearly as strict in their division of work between the two pitches, but it’s a general trend across pitchers.
- Sliders tend to be thrown more often to same-handed hitters.
- Changeups tend to be thrown more often to opposite-handed hitters.
Expanding on the idea we touched on with Soria, a slider can be a very different animal from one pitcher to another. Some sliders look almost like cutters (John Smoltz), some look almost like curveballs (Joakim Soria, Francisco Rodriguez), and others have only a little spin-induced movement (Brad Lidge, Jeremy Bonderman, Felix Hernandez). Yet all of them are called sliders, and all (except for Soria) are well-known as among the best sliders in the game. I’ve been attempting to gain a better understanding of the slider in order to get a better handle on how to classify a pitch with so many sub-species.
One thing I’ve learned, thanks to John Walsh, is that sliders without much spin-induced movement–such as those we identified earlier in the Gagne analysis–have, in addition to a low x-z component of the spin rate, a great deal of error in the calculation of the direction of the spin axis. Such sliders have a spin axis pointing nearly in the direction of travel (the y direction), and any x-z component of the spin axis is small and highly susceptible to measurement error.
- Spin direction calculations have a high degree of error for sliders with very small spin-induced movement.
Another pitch that deserves a quick mention is the gyroball. What is it? I know of one pitcher who definitely throws the gyroball: C.J. Wilson. I even have identification by Wilson from his own blog of a few specific game instances when he threw the gyroball. From my previous reading, I had expected the gyroball to be slider-like, but Wilson’s gyroballs look like fastballs, so much like fastballs that I have been unable to identify them as an otherwise separate pitch from his fastball. This is an area for further investigation.
I’ve also been investigating a few new ideas in pitch classification, including Tom Tango’s suggestion of “reaction break” and adjustments to the spin-induced movement and spin rate calculations based on new formulas from Alan Nathan. Both of these show promise, but I’m still in the early stages of investigating their application.
In summary (at last, you say?), we’ve come a long way in the science of pitch classification since spring 2007. We have the invaluable reference of Josh Kalk’s player cards. We have a sophisticated set of tools for classifying the repertoire of an individual pitcher. However, we are still not near the point of having a universal system for classifying >95% of major league pitches accurately. A published system with that level of accuracy would serve as the spark to ignite a great deal of other research that could transform the game of baseball.
Perhaps Sportvision and MLBAM are developing such a system in private. Perhaps Joe Sheehan’s unpublished system is closer to that goal than I realize. Perhaps Josh Kalk is on the way to making refinements to his system that will get us closer. Maybe John Walsh’s approach, which went to press as I was writing this article, of using K-means clustering and going through the major leagues pitcher by pitcher is the right way to start.
Clearly, one of my priorities should be improving my understanding of and facility with clustering techniques. In my opinion, an effective clustering algorithm should include speed, spin direction, spin rate, vertical spin deflection, horizontal spin deflection, and handedness of the pitcher and hitter. It’s possible that additional variables such as strike zone location, release point, and ball-strike count might be helpful, but I have a feeling their addition to a clustering algorithm might cause more problems than they solve.
I would like to develop a classification system that would be available to any researcher and accurate enough to distinguish splitters from changeups, sinkers from four-seam fastballs, and cutters from sliders and fastballs. I confess to being a bit daunted by the task and not very experienced in some of the statistical math that may be required, but I’m going to keep chipping away at this problem until we get there.