The triumph of Pythagoras
October 14, 2007 8 Comments
On the SABR Statistical Analysis Listserv, there’s been a great deal of chatter concerning the good old Pythagorean win estimator. This year, as it seems happens every year, most teams finish around their estimates. But, there always seems to be that one oddity and this year, it’s the Arizona Diamondbacks. The Diamondbacks were outscored this year (712-732), and had a Pythagorean expectation around 79 wins, depending on exactly which formula you use. They won 90 games, good for the best record in the NL. Huh?
So, are the Arizona Diamondbacks a sub .500 team, like their Pythagorean projection says or are they a 90 win team like their… ummm… actual record says? It’s an interesting question. When trying to figure out how “good” a team is, which should we look at? This is a topic which has been taken up before by Chris Jaffe, specifically with reference to the Diamondbacks, and more theoretically a few years ago by Dan Fox. Dan found that early in the season, if you want to know what a team’s season-ending winning percentage will be, you’re best to look at their Pythagorean record. That is, until about 100 games in, when the team’s actual record becomes the better predictor of their season ending record. (By the end of the year, actual record is a perfect predictor of season-ending actual record.) But which one better predicts what a team will do in its future games?
In July of this year, Joe Sheehan of Baseball Prospectus made the assertion that “Run differential is a key measure of team quality, and a better predictor of future performance than win-loss record.” Well now, sounds like something we can test. I took the Retrosheet Game Logs from 1980-2006. (666, no kidding, team-seasons) I took each team’s games in sequence. After each game, I calculated the team’s actual winning percentage, as of that moment, as well as their Pythagorean projection as of that moment. So, if a team is 10-10 after 20 games and had scored 93 runs while giving up 91, I ran the numbers. (Methodological note: I used the David Smyth/Patriot formula and the standard formula with a 1.82 exponent, although they were pretty indistinguishable, so I just reported the Smyth formula) Then, I calculated the team’s actual winning percentage over the rest of the season. So, if that team went 72-70 over the last 142 games, I calculated those numbers. I ran the numbers 162 times, one for each game of the year. Which of the first two (current actual win percentage or current Pythagorean projection) was a better predictor of performance over the rest of the season from that point forward?
Want to see a pretty graph?
The graph shows correlation coefficients of the two methods to performance the rest of the way. Coefficients are low at the beginning of the season because after game one, everyone’s either got a winning percentage of 1.000 or .000, and that’s not going to correlate well with much of anything. At the end of the year, there’s the same problem in the opposite direction. Focus on the middle part of the graph, where the sample sizes in both halves are roughly equivalent. That’s where the story is. You’ll see that the green line, representing the Pythagorean projection (using the Smyth method, although the 1.82 method had the same pattern) at that particular moment is consistently above actual winning percentage. At the exact midpoint of the season (81 games), Pythagorean projection correlates with winning percentage the rest of the way at .494, while actual winning percentage has a correlation of .464.
(Side note: The weird jump around game 110 is because of the 1981 and 1994 seasons. Teams played a little less than 110 games in those years, which led to some funky data in those years… just enough to cause a little blip in the data.)
In terms of predictive power, run differential really is the more important information to know when it comes to predicting the future. What’s the deal with the Diamondbacks? Well, for what it’s worth, the correlation between Pythagorean projection and future performance at 81 games is .5, which isn’t bad, but it isn’t all that great. In fact .5 is possibly the most infuriating correlation coefficient out there. .5 means that about 25% of the variance is explainable by whatever factor you’re using as a predictor. 25% is a quarter of the variance! But 25% is only a quarter of the variance. As the season wears on, the gap between Pythagorean and actual win percentage narrows, until they become roughly the same around game 150 or so, where the correlations are around .35. The thing is that at game 150, the sample size for the “rest of the season” is only 12 games, and by that point, Pythagorean projection and actual winning percentage are usually mirroring one another.
But, there’s evidence here that a team is better described, over the long run, by their run differential than their actual record. This will certainly come as great news to fans of the Padres and Braves, who finished with the 2nd and 3rd best Pythagorean win percentages in the NL this year, as they watch the Diamondbacks in the playoffs this year.