How Can We Measure the Quality of Play?

Last week, David Gassko (you might remember him, he used to write on this site) published an article on The Hardball Times attempting to measure the changing quality of play, or the difficulty a hitters faced over the history of baseball. The approach is to look at what a player hits one year, then see what he hits the next. David incorporated regression to the mean in his analysis, an important step, as if you don’t account for it you will greatly overstate the improvement in quality of play.
Overstating the improvement over the years can lead to erroneous conclusions, such that Honus Wagner playing today wouldn’t be any better than Neifi Perez, or that Babe Ruth would be no better than, I don’t know, somebody like Kevin Mench. While those assertions don’t seem right, we can’t prove them since Wagner and Ruth are dead and can’t show us what they would hit today. We do know that the league hasn’t improved by quite that degree, because the same rate of improvement would mean that an average hitter (think OPS+ = 99) from 1985 would not be able to play today, and we have fossil evidence to disprove that, namely Julio Franco.
Gassko’s conclusions seem to provide a reasonable, conservative estimate of the improving quality of play, but there remains a problem. What about age? If the majority of players are 27 or older, then they are past prime, and will be declining as a group. The study must be adjusted for age.
Now there’s the problem. How do we know what appropriate age adjustments are, and when a hitter reaches his peak? By performing the exact reverse methodology that was used to estimate the change in league quality.
The age and quality of play adjustments cancel each other out. If we cling to peak age for a hitter = 27, then mathematically, we have to conclude that there has been no change in the quality of play for pretty much all of baseball history.
There is a great discussion of this subject on The Book Blog .
What we are left with is this: Looking at player changes from year x to year x + 1 can tell us the combination of effects of age and effects of quality of play. Actually, even more than this, ballparks are changing as well. What this type of analysis cannot do is separate the effects.

Advertisements

5 Responses to How Can We Measure the Quality of Play?

  1. John Beamer says:

    If we cling to peak age for a hitter = 27, then mathematically, we have to conclude that there has been no change in the quality of play for pretty much all of baseball history
    ***
    Sean
    I’m not sure that is 100% true. Doesn’t it depend on the shape of the age curve? Suppose every year a 21 year old become 0.01 wOBA better than a 21 year old the previous year. If you assume a age 27 peak with a standard parabala then cumulatively you’ll see talent levels rise.
    Agree on the point that this is impossible to separate in the data — I think David has a follow-on piece on Thursday, so will be interesting to see what he says

  2. Sean Smith says:

    John,
    I think if true peak = 27, and players were improving by such a slight margin, then our observed peak age, which is true peak + talent improvement, would be 26, or maybe 26.9, but something a bit less than 27.

  3. Pizza Cutter says:

    I always find these “How would Babe Ruth hit today” discussions a little odd. If he were to be transported from 1927 Star Trek style into Yankee Stadium tomorrow and told that he was hitting cleanup, it’s one question. But, suppose we went back and found Ruth’s DNA and cloned him in 1980, and raised him so that he would now be a 27 year old, then stuck him in the middle of the Yankee lineup, it’s another.
    Also, just looking at raw performance numbers can be misleading. Had the ’95 replacement players actually taken the field in actual games and played a whole season, the “quality” of play would have been much lower than the “real” players, but would the stats have looked roughly the same?

  4. Sean Smith says:

    We could have detected a few things about the quality of play of those guys. Pretty much the same things we get by looking at the low minors – higher error totals, large standard deviations in other performance statisitcs.
    Plus, if a few players crossed the lin – say Lenny Dykstra played against them and hit .380 – we’d have another data point telling you the league is inferior.
    With Ruth, I’m not sure what we mean either, time travel or cloning to raise him in the 80’s-90’s America. But either way, he would be competing against players drawn from a much larger population of eligible players.

  5. John Beamer says:

    I agree with PC … these debates are a little facile, but fans are obsessed with them. We always preach that baseball is about context and Ruth and Bonds played in two vastly different eras with a different context. For the reasons that Sean (and others) have outlined I find it impossible to compare them statistically. There are some facts to support assertions like “The competition that Ruth faced was inferior to that faced by Bonds” …. However, the question of if we could transport Ruth to today how would he do is, imo, unanswerable
    Sean — I hear you on the age curve and I *think” you are right (no slight on you, I just can’t get my head around it at the moment) … I do think though you could maintain a peak age of 27 by adjusting the back end of the age curve (ie, it falls less steeply), but you’ll soon get to a point where you can’t do this (or the peak age shifts forward).
    Interesting discussion anyway.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: