MLE's, how useful are they?
June 11, 2007 9 Comments
Do MLE’s (Major league equivalencies of minor league stats) predict future major league performance as well as major league stats? Last winter I compared the results for several projection systems (CHONE, ZIPS, Marcel) for all hitters with at least 300 at bats in 2006.
I took this list, and determined which players had a substantial part of their projection coming from minor league stats. Generally if a player had less than one full season in the majors, I marked him as an MLE player. While the CHONE and ZIPS projections use MLE’s, Marcel does not, so a player with no MLB time, his projection will be the league average.
With all players who had at least 300 at bats in 2006, the correlation for projected OPS and actual OPS was between .61 and .62 for each system. Taking out the low experience players, the correlations jump to the .64 to .65 level. But using only the MLE players (41 of them) the correlations are .38 (ZIPS) and .34 (CHONE/Marcel). They aren’t useless, but do not predict as well as major league stats. Marcel’s correlation would be zero if we used players with no major league experience, using only the league average, but did pretty well using a half season or so of major league performance for this group of players.
Its possible that the sample size was low, and this was just a bad year for predicting rookies. Also, there are selective sampling problems, in that rookies who play well get to keep playing, and those who do not are more likely thanplayers with a strong MLB track record to get sent back to the minors. This doesn’t seem to be a huge problem though, as among this group the average actual OPS was only 3% higher than average projected OPS.
The players who CHONE missed the most on:
1. Ryan Howard. Everyone missed on him, the only question is how much. Howard’s MVP season was likely over his head.
2. Brian McCann. Same story. He was supposed to be good, especially for a catcher, but not THAT good.
3. Dan Uggla. MLE’s said he wasn’t very good at all. He is.
4. Andre Ethier. Another one the minor league stats didn’t think would be so good.
5. Ronny Cedeno. The first one on the list who played worse than his projection. If he wasn’t a shortstop, he would not have gotten so many at bats, and would not be part of the sample.
6. Ryan Zimmerman. Actually, ZIPS nailed this one, CHONE didn’t think he was quite ready. Marcel gave him a good projection simply based on 58 great at bats in September 2005.
7. Jeremy Hermida
8. Hanley Ramirez
The Marlins really screwed up the whole MLE process, didn’t they? Uggla in 2005 played in AA and hit .259/.302/.354 Ramirez in AA hit .271/.335/.385 They played well. Hermida hit (in as good of a pitcher’s league as the other 2) .293/.457/.518 at the same level. So of course Ramirez and Uggla are the ones who hit at the major league level.
Players that CHONE projected very well, within 10 points of actual OPS, were Chris Shelton, Jose Lopez, Shane Victorino, Conor Jackson, Matt Murton, Jose Bautista, Yuni Betencourt, and Ian Kinsler. I didn’t predict the hot start and later slump of Shelton, but at least got the final batting line down.
Minor league stats should not be ignored, but it appears they are not as predictive of major league stats. Perhaps we can improve projection systems by weighting them less than major league performances.