Visual WPA Results
April 24, 2008 5 Comments
Two weeks ago I discussed a version of WPA in which intuitive scouting would aid in the in-depth division of contributions between batter/runner and pitcher/fielder. Though not a new or revolutionary technique I felt it was worth bringing up to serve as a potential measure of the human aspects not currently found in the WPA statistic. Essentially, if those against stats like WPA feel that its results are tainted due to a lack of division amongst the true efforts in each play–situations like a tremendous fielding or baserunning play being solely credited to the respective pitcher and hitter–then incorporating said division would theoretically produce different and more accurate results.
In conducting this analysis I used the 4/21-4/22 series between the Phillies and Rockies. Though I definitely agree with Pizza Cutter’s assertion that a group of different eyes determining the credit or debit division is a better idea, the judgments here were left up to my own eyes. In no way did I attempt to cherrypick any data to prove a point; this was merely done to investigate an idea. I watched every play with eyes that have seen a ton of baseball games, often watching plays a few times.
The key in comparing results is understanding that we cannot jump from Point A to Point C. The current WPA does not divide contributions; if we compared those figures to the in-depth play divisions common sense suggests drastically different results will be found. Before going in-depth WPA needs to be adjusted to divide credit or debit between, at the very least, errors. With this in mind I broke the analysis into two steps: First, just separating contribution amongst amazing/bad plays unfairly credited or debited fully to the wrong person; and secondly, dividing contribution on a deeper level, gauging things like outfielder distance, strength of some arms, would your average runner reach third base, etc. For instance, Chase Utley’s web gems in this series would be included in Step 1 as well as Step 2 whereas something like properly determining the debit recipient on a Wily Taveras stolen base would apply solely to Step 2. This way, we can compare the current WPA to Step 1 and then Step 1 to Step 2; this comparitive system will be more effective in determining just how different the results may be.
Here are the links to the Fangraphs WPA for these two games, as well as a PDF showing the total WPA for the series:
Here are the links to the VisPA results:
Of the 170 plays in this two game set, a total of fourteen were effected by the Step 1 and 26 were altered via Step 2. Despite only 15% of the plays requiring some type of adjustment there were noticeable shifts in the WPA of certain players. Chase Utley, for instance, came in at +.454 via standard WPA; both his VisPA1 and VisPA2 were +.589. Because of his tremendous fielding plays he increased somewhat significantly. Pat Burrell, on the other hand, had a standard WPA of +.674; his VisPA1 was +.512 while his VisPA2 was +.333. Due to poor fielding and certain plays benefiting from heads up baserunning on the part of others, Burrell’s standard WPA significantly decreased from WPA to VisPA1 and dropped off even more from VisPA1 to VisPA2. Here is a file showing all three types of WPA for everyone in this series:
Something I ended up doing, which I’m curious to hear thoughts on, was treat a certain play in a fashion similar to inherited runners. It was an error made by Pat Burrell that put two men on prior to Yorvit Torrealba hitting a 3-r homer. If Burrell makes that play, the inning ends; since he didn’t, and three runs scored on the home run, I charged him with 1/3 of the WPA debit on that play. This is not necessarily something I fully advocate but something to consider and generate feedback on. It was only one play in this short series so the end result isn’t too significant. There were no instances of an umpire effecting the outcome of a play with a bad call and no signs of wrongdoing by the third base coaches either. Some plays were divided based on the difficulty level of properly executing; others, as in Utley’s web gems, were awarded entirely to the fielder.
Based on standard WPA, the most valuable player of this series was Pat Burrell. Using VisPA1, and in this case VisPA2, Chase Utley is the most valuable player. If you asked anybody that watched the series who they would pick as the MVP it would likely be unanimous in Utley’s favor.
This just reinforces that much more can be gathered from mixed methods in sabermetrics. For all we know, over the course of a season, everything could even itself out to the point that standard WPA is 90%+ accurate. I’m still very intrigued by the idea of putting something together across the web to track this for an extended period of time. Even if the results end up cancelling each other out in the aforementioned scenario I feel like we owe it to ourselves to try. After all, by using a group of eyes to evaluate the proper debit and credit on specific plays meriting said division, we will incorporate human aspects of the game not found in a play-by-play file or game log, and in turn offer a more accurate measurement of what we are seeking to measure.