Table of Contents

Tuesday, October 23, 2007

Player Value, Part 2c: Offense - Positional Adjustments

To view the complete player value series, click on the player value label on any of these posts.

Positional Adjustments

If you go back and compare the RAR numbers in my baselines post to BPro's VORP, one of the biggest differences that you might note is that players playing catcher, shortstop, and second base tend to get higher ratings in VORP than in my numbers. The opposite is true for first basemen, left fielders and right fielders. The reason? VORP includes an adjustment based on typical offensive performance at each position, and my data did not. ... but should they have? Let's take a look.

In the table below, I've grouped NL hitters from '03 to '07 by their primary position and sorted them by their rate of offensive production (runs per game). Here's how those totals break down:
POS PA R/G %LgAvg OBP SLG OPS
C 43228 4.1 82% 0.319 0.393 0.712
SS 46332 4.4 87% 0.326 0.401 0.727
2B 49834 4.7 94% 0.338 0.414 0.751
CF 45720 4.9 96% 0.335 0.424 0.759
3B 47791 5.2 104% 0.344 0.449 0.793
RF 46754 5.4 107% 0.348 0.454 0.801
LF 48358 5.6 111% 0.353 0.465 0.818
1B 49581 6.0 119% 0.364 0.483 0.847
The rankings here probably don't come as a big surprise to many of you--after all, it's often the case that teams will employ catchers and shortstops that aren't very good hitters (e.g. David Ross and Alex Gonzalez with the Reds). First basemen and left fielders, on the other hand, are often some of the best hitters on any given team. But why is it that the positions vary in their hitting so much? I can think of two possible reasons:
  1. A poor hitting position may be a more "difficult" position to play and thus the pool of players who can competently play that position may be smaller than at a position that is easier to play. Smaller pool of players = less offensive depth.
  2. The players playing a weaker position may not be as talented (defined as total offensive + defensive skill) as those at other positions.
The first is almost certainly true. We expect shortstops, for example, to have tremendous range, great hands, and a canon for an arm, because those attributes are what is needed to get lots of outs at that position. The same expectations are not present for first basemen, and therefore there's a larger pool of players that can play first base...and thus, there's a higher standard for offense at that position than at shortstop.

Therefore, analysts often make positional adjustments to offensive performances based on these observed differences in hitting across positions. The logic is that if teams play players in appropriate positions relative to their defensive skills--and, in general, they probably do--we can give a boost to poor hitting positions and a penalty to plus hitting positions such that the average hitter at each position is given equal value. That way, each player is compared to his peers, putting everyone on an equal playing field. Presumably, in terms of total player value (offense + defense), if a player moved from first base to shortstop, any gains he would make in terms of being a better-than-his-peers hitter would be negated by the cost he would incur to his team via substandard defense.

That's essentially the basis for positional adjustments in VORP. However, there's a major flaw in that approach. And it relates to the second explanation for variation in offense across positions, that of variation in talent level across positions.

Why positional adjustments based on offensive disparities is not the best approach

Let's come at this from a fresh angle: players aren't restricted to one position, but can theoretically "play" anywhere on the baseball diamond. The problem is that some are better defenders than others, and therefore teams tend to put their best defenders at the positions that are the hardest to play (both in terms of the physical demands of the position, and the level of average defensive performance at that position), such as shortstop. But how much harder is it to play shortstop, for example, than first base?

The best study that I'm aware of that has tried to quantitatively answer this question is one by Tom Tango. He used multi-season UZR data to compare how players performed when they played multiple positions. Presumably, a player's absolute defensive skill is a constant, but he'll look better or worse at a position given how he stacks up to his competition at each position (especially once you adjust for experience at a position). By comparing how player defense varied across different positions, in virtually every combination and direction you can imagine, Tango constructed the following defensive "spectrum" (it should still be considered a work in progress):
+5 CF
+4 SS
+0 2B
-1 3B
-4 LF
-4 RF
-8 1B
The units are the typical differences he found in defensive runs saved per season that you should expect when a player moves from one position to another. So, if you move an average fielding first baseman to shortstop (assuming you can do this, i.e. he's not left-handed), you should expect that player to play roughly 12 runs below average per season once he learns the position (players will vary, of course, in how they do based on their specific attributes--speed, arm, hands, etc--these are just the mean differences).

Here's another look at this question, based on data can be gathered from the Fans Scouting Report, which asks fans to rate players in a variety of categories based on their subjective impressions of a player's skills (participants are asked to ignore player position, as well as any defensive stats...hence the "scouting" report). Here are the average total scores for players at each position, all of which are scored in the same categories:
CF 60
SS 59
2B 56
3B 54
RF 50
LF 43
1B 41
The average player across all of these positions got a score of ~52. Therefore, we see that players who play center field or shortstop are generally given substantially above-average ratings on their skills (speed, first step, hands, arm accuracy, arm strength, etc), whereas players in left field or first base are rated as below-average defenders. While the actual numerical differences aren't the same (it is possible to convert differences in these ratings to an approximate runs saved statistic), the overall positional rankings are almost identical to those that Tango generated using UZR data. The only difference is RF'ers being ranked as better than LF'ers. Pretty compelling when two such vastly different datasets come to virtually the same answer!

Now, looking at these two defensive spectra, they look pretty similar to the offensive rankings, right? Shortstops are ranked near the top, while corner outfielders and first basemen are down at the bottom. That's certainly supporting our first explanation--some positions are harder to play defensively, and therefore the pool of players that have sufficient defensive skills to play those positions competently is smaller, meaning there's less offensive depth at those positions.

On the other hand, there are some notable differences between the offensive and defensive positional rankings. Center fielders, for example, were rated as approximately average hitters (96% of league average), and yet were rated as the single hardest position in baseball to play in both datasets. So here we have a position that features players that are average hitters as well as above-average fielders. This means that center fielders are, overall, an above-average position in terms of total player talent! On the other side of the coin are second basemen. They are rated as an average position, defensively, and yet also feature below-average hitting. This means that second basemen, overall, are a position with below-average talent (again, talent = combined offensive + defensive skill).

Now, think about what this means if we use adjustments based on offensive disparities among positions. If a player is at second base, and then is moved to center field, he is moving to a position that is more difficult to play defensively. At yet, because center fielders, on average, tend to hit better than second basemen, he is going to be judged against a higher offensive standard than he was at second base. Therefore, if we use positional adjustments based on offensive disparities, this player is going to suffer a hit to both his offensive and defensive ratings just because he moved to a more talented position!

To put it another way, positional adjustments based on offensive disparities assumes a negative correlation between fielding skill and offensive performance. The fact that we have positions where the average player at that position is both a superior defender and an average hitter means that this assumption cannot be correct.

Therefore, if the point of the positional adjustments is to put all players on an even playing field with respect to our value estimates, it seems to me that adjustments based strictly on offensive variation across positions are inadequate. They will underrate players playing particularly talented positions, and they will overrate players playing talent-poor positions.

To be clear, there is absolutely a need to apply positional adjustments to player value ratings, because the average fielding skill (and thus value) varies across positions: an average-fielding first basemen as far less of a defensive asset than an average-fielding shortstop, and we need to recognize that if we're going to value our shortstops and first basemen appropriately. It's just that using differences in offensive performance is a flawed way to go about this, because overall player talent levels are not constant across positions. Instead, we should be using adjustments that account for differences in the actual fielding value of average defense at different positions, like Tango's UZR spectrum above.

So all that said, here's "my" solution on how to assess player value (it's most certainly not my idea, just what I've come agree is the best way to go about things):
  1. Estimate player offensive value. This can be done relative to overall league averages or replacement level. It should be done without regard to position: in this step, we're strictly interested in offense, so it doesn't matter where the player plays!
  2. Estimate fielding value via two steps:
    1. First, calculate player fielding value relative to overall league average at a player's position (as we saw in my replacement level study, this works for both league-average and replacement-level baselines).
    2. Assign a prorated (by inning) positional adjustment to each player that accounts for the differences in fielding value across different positions. Using Tango's data, a first baseman would be get -8 runs per season relative to the rest of the league, whereas a center fielder would be rated at +5 runs per season, relative to the rest of the league.
  3. Sum all these values together to get a composite estimate of player value.
We know how to do #1 already. The next article(s) in this series will discuss how to do both parts of #2 in detail.

Update: After some additional data analysis and discussion, Tango and others seem to have moved to this spectrum, which is slightly (emphasis on slightly) different from that which I've been using. Numbers are runs per season:
+10 C
+6 SS
+4 CF
+1 2B/3B
-6 LF/RF
-9 1B
-15 DH

This is what I'm going to use moving forward. But it's close enough that I'm not going to revise anything I've done to date. Sometimes you just gotta move forward! :)

And here's more on the apparent fact that the assumption of positional equality in offense+defense is a flawed assumption.

Update #2: As with any field, this research is constantly evolving. Based on several rounds of further discussion and analysis, the current positional adjustments endorsed over at TheBookBlog are:
+1.25 C
+0.75 SS
+0.25 2B/3B/CF
-0.75 LF/RF
-1.25 1B
-1.75 DH
Multiply these by 10 to convert wins to runs and you get the adjustments I'm currently using. I probably should really be multiplying by 9.5 or so (4.75 r/g * 2 teams = 9.5 r/g), but it makes almost zero difference and the round numbers are easier to remember. :)

22 comments:

  1. That is a thought-provoking concept. I want to make sure I understand this correctly. If you move a league average LF to 3B, then he would field 3B at 3 runs below the league-average third baseman? Is this correct? Thanks.

    ReplyDelete
  2. On average, after he learned his position, yes, our best estimate is that he'd play ~3 runs below average defense at third base.

    Mileage will vary, of course, depending on the various skills of the player (arm strength, reaction rate, sprint speed, hands, etc). But that's the estimated average change in defensive ability, based on Tango's study.
    -j

    ReplyDelete
  3. That defensive translation stuff is very interesting, but I wonder how translatable it really is. Averages can be useful but can also be very misleading for comparisons of either broad or skewed distributions.

    The issue of skill set difference is discussed, but not incorporated. I think it's extremely problematic. Adam Dunn converts to a better SS than Hanley Ramirez. Also, BJ Upton rated as a minus 2B and a plus CF.

    Between skill set types, real learning curve, defense ability regression due to age, etc. this has a long long way to go. Definitely interesting stuff -- but virtually useless at this point.

    ReplyDelete
  4. I dont recall ever seeing an argument against Woolner/BP's VORP positional adjustment that was this clear-cut. Justin, this evaluation of widely used principles is incredible.

    A minor gripe, however, is that the UZR and fan's fielding data is measured against average. In previous posts you said two things:
    1) Comparing to a replacement player gives better insight into the value of the player than comparing to average

    2)Replacement level fielding is relatively similar to league-average fielding. Even if it is the same as the average, the name should at least be called a baseline.

    Overall,great job

    ReplyDelete
  5. one last thing, the bold font in the essay is missing an "are" before the word "inadequate"

    ReplyDelete
  6. Rick,

    Well, needless to say, I completely disagree that these values are "virtually useless." Can they be improved upon? Sure, of course they can. If nothing else, it might be nice to give right- and left-handed throwers their own spectra (that somehow inter-relate), since left-handed throwers are so restricted with respect to where they can play. But I still feel that this approach is a step forward for our ability to rate players.

    The fact that you can find players that violate the typical value differences is not a massive problem for this method. For one thing, defensive stats are about twice as volatile as hitting stats, and therefore we're always dealing with substantial uncertainty when looking at any particular player.

    Furthermore, on some level, I think we have to reward teams for playing players in the positions where they will perform best. What we--or at least I--want to do with a positional adjustment in terms of player value is account for the typical differences in defensive value of a player at one position versus others. After all, from the perspective of offense, a hit is a hit and a run is a run. It's defense where position matters, and an average fielding left fielder is not worth as much as an average fielding shortstop because it's easier to be an average fielding left fielders than an average fielding shortstop (this analysis says the difference averages at around 8 runs).

    Does it mean that every player will gain or lose 8 runs per season on defense when they move between these two positions? Of course not. But that's the typical change we've seen, and therefore the typical difference in defensive value of a shortstop vs. a left fielder.

    To me, even with its uncertainties, this approach is far more preferable than the use of position adjustments based on offensive disparities. It recognizes that there may be differences in talent levels across positions, and seems to me to more closely recognize what actually is different about players at different positions. -j

    ReplyDelete
  7. Hi Dan,

    Thanks--like I said, it's not my idea, is just what I've come to agree works best. Overall, perhaps it doesn't make a huge difference. But at shortstop and center field, in particular, it's important.

    Regarding your point about replacement level...I see how it might be confusing. Replacement level fielding IS league-average fielding. The reason I shy away from calling it replacement level fielding, however, is that BPro's replacement-level fielding stats use a baseline that is much lower than average. And I don't want what I'm doing to be confused with what they do, because the data I've seen indicate that their definition of replacement level fielding is not meaningful (it actually puts the total value of a replacement player much lower than anything the resembles reality...that's one of the issues that you run into with WARP).

    Thanks also for the typo--I'll fix when I get a sec. I'm always omitting words...apparently I go back and edit too much, even on my proofreading pass(es).
    -j

    ReplyDelete
  8. Perhaps useless was a bit blunt. (and please don't confuse my criticism of that point with my love of this series you're doing)

    I'm just not sure how you can use those numbers in a valuation process due to the extreme amount of variation around those means. While the average is meaningful as an average, the amount of variation is such that to apply it to any specific player simply adds to much error. It's certianly useful in understanding the relative "talent" as you put it, but not as useful in player valuation, which is really the heart of your series.

    Presumably, (with all due caution) you would take this to the point where using an adjustment such as Tom Tango's, we can say simply that Dunn is a -22 runs defender (-18 + -4). Thus, we can also compare him to all offensive players and create a position-neutral player to player valuation system.

    I am curious, is that scale comparable to the plus/minus one you've used? Especially at the extremes, we see cases where great firstbaseman would be good SS and bad CF would be just mediocre 1B -- and neither of those seem very intuitive.

    ReplyDelete
  9. Hi Rick,

    I don't know...do you think there's more variation around these means that there is around the offensive means at each position?

    In other words, the traditional approach is to use an 87% adjustment for shortstops because they, on average, hit that much worse than average. But there's obviously a huge amount of variation around that number, because some shortstops are comparable with just about any hitter in baseball, while others are among the worst hitters in baseball. So is that approach any better in terms of applying means to individual players? I'm not sure that it is. And yet, folks seem comfortable using those numbers as estimates of position value. ... despite the fact that it doesn't really measure what we're trying to measure in positional adjustments (differences in defensive value among positions).

    As for the issue of how good a first baseman could be...I don't know. Albert Pujols is the best defensive first baseman right now, and is a heck of an athlete. I'm not sure that he wouldn't be a decent shortstop.

    And Griffey was certainly an awful center fielder. Do you think he'd be much better than average at first? Somehow I doubt it.

    Again, I'm not arguing that these numbers are perfect. But I think that this approach is the approach we should take in adjusting player value by position, at least as much as possible (catchers and DH's are a significant problem...). If we don't use those numbers, what is the alternative? I see two: make no adjustment, which is obviously unhelpful and ignores what are clearly differences in value among positions. Or, to do an offense-based adjustment, which seems completely inappropriate to me given what we know about relative position talent.

    Maybe there's an approach that falls somewhere in between, but I don't have a good idea of how to objectively come to that point. We could figure the average runs difference based on offensive disparities, and then average that difference with numbers from Tango and the Fans scouting report.. That might be a worthwhile approach to take (sort of a best-and-worst of both worlds approach), but I'm trying to avoid sticking my neck out too far for now until I've at least mastered what folks have done to this point..
    -j

    ReplyDelete
  10. Justin, I think I used poor phrasing to make my point. It's not that defense is more variable. It's that the offensive distribution is substantially more right skewed. Given a group of equal defensively talented players, the variation of offensive of talent as measured by is quite spread - a greater standard deviation in terms of run production. Given a group of equal offensive players, the run production distribution is tighter.

    We can assess this pretty easily. What is the average run production differential in offensive production between a 1B and CF? What is the average run production in defensive production between a 1B and CF (13 runs according to Tango)?

    Without seeing the numbers (your posted offensive and defensive numbers aren't on the same scale), I'm going to assume for the sake of this argument that the offensive difference is greater.

    Offensive production is offensive production regardless of what spot in the batting order it comes from. That is to say, each PA is created equal. You can't "hide" a batter to minimize the effect of their offensive ability in the same way you can hide a poor defensive player. At most you're looking at a batting order difference that results in about 10% fewer at bats for the poor hitter. For fielders, there's not only an issue of the quantity of opportunities, but the quality of them. Surely, the talent required to field a ball in LF or 1B is different than at SS.

    Secondly, the defensive baseline for playing the field in the major leagues is much much lower than for hitting. The bare minimum amount of both offensive and defensive production has been established over decades. There is some bare minimum level of talent that non-DH, non-pitcher must have. Guys like Jack Cust and John McDonald are at either end of this spectrum. If Cust was an even worse fielder (or if he had to play the field every day) or if McDonald was any worse with the bat, they wouldn't be viable major leaguers because their lacking skill set would too greatly offset their strong one -- and their strong one is essentially as good as is possible.

    Put another way, there are many 1,000s of players who have the overall position-neutral defensive talent necessary to play in the majors. There are less than 500 people who can hit well enough to be in the majors. This is in part why the DH was created. There are few Frank Thomases or Edgar Martinezes and many many Juan Castros and Neifi Perezes.

    I guess my problem is coming up because of this -- given the relatively small difference between positions, hasn't the market overvalued defensive production?

    It would seem that given that league average offensive production is harder to come by than league average defensive production, that we should see crappier defense than we do. Are we putting up with way too many crappy hitting guys who aren't substantially awesome defenders? Why would we tolerate a .700 OPS SS with a great glove but never tolerate a .700 OPS LF regardless of his glove, when the defensive difference due to position is just 8 runs?

    It would seem that one of 3 things is going on:

    1.) I'm over-estimating the spread of offensive production. That is an 8 run spread is more valuable than I think it is.
    2.) A lot of GMs (ie. nearly all) are overvaluing defensive production relative to offense.
    3.) The translations suggested by Tango understate the positional differences.

    Sorry for the rambling post. I realize this isn't really a direct response to my earlier comments and your response to those -- just thinking out loud.

    ReplyDelete
  11. Hi Rick,

    Lots of great points in there.

    A few comments:

    * First, offensive disparities among positions are indeed larger than Tango's numbers, and sometimes are much larger. For a rough estimate, 1 r/g (r/27 outs) is worth about 15 runs a season.

    Reasons for this are many, but here's a few ideas:

    1. Position talent varies more than we probably typically think it does. I really think this may be a huge part of it. After all, the most talented (offensive+defensive skill) players usually start at either shortstop or center field and then move "down" the defensive spectrum as they move up through the minors, right? That is generally going to mean that the most-bestest talented players will be those that stick at shortstop and center field.

    2. There is probably some selection bias in Tango's data, because it can only collect information on players that are actually allowed to move positions. The estimated difference between left field and first base, for example, may be the result of tracking how 1B/LF's do at those positions, and then how utility infielders do when plugged in at first base (or, how SS/3B's do, and then how 1B/3B's do). Those are very different groups of players, which adds some uncertainty between the numbers... especially because, as you mentioned, the skills required to do well at a position do differ among positions.

    When it comes to actually trying to evaluate these skill differences, I think the fan scouting report might turn out to be our best available tool. Something to add to my list of things to look at..

    FWIW, depending on the groups you select, Tango found that there is evidence that all outfield positions are substantially harder to play than infield positions. I don't buy that, and neither does he, which is why he ultimately went with the spectrum I cited.

    Nevertheless, I still don't see that measuring offensive disparity really tells us what we want to know when making position adjustments. The fielding-based adjustments seem much closer to the mark, because they're really trying to measure what is actually different among positions, even if the results are more conservative.

    * Second, I do think that a lot of GM's (and fans) don't value offense vs. defense as well as they could. I think many seem to weight offense higher in some positions, but defense higher in other positions. That's why one of the first things I tried to do in this series was illustrate that a run saved is just as valuable as a run scored. And if that is the case, then we should be willing to accept any combination of offense and defense that provides above-average value. That goes for left field as much as it goes for shortstop. The adage of being willing to sacrifice defense in corner positions for the sake of offense doesn't make sense--except that you better-leverage poor defensive players by putting them in positions where they won't get as many chances (and Tango did try to account for variation in chances in his data).
    -j

    ReplyDelete
  12. Justin, I think that perhaps I'm thinking more in terms of some sort of real limit on available talent by position.

    If we were to limit the possible SS to just those people who could play it at -20 runs (as managers essentially do in practice), that would limit the pool of potential SS, excluding somebody like Adam Dunn. I would suggest that as we move up the chain, the average offensive production from the true available pool decreases.

    This sort of selection naturally occurs on the offensive side, as no player who OPS's less than say, .600 is given a full-time job -- because we "know" that no amount of defense can make up the difference between the overall value of a free available lesser defender who's better with the stick.

    So, out of the 1,000 theoretical possible SS (including Adam Dunn), only 100 of them can field the position within 20 runs of average. Any person who you want to count as a true replacement must match that bare minimum defensive threshold.

    Where this really matters is when we use the concept of replacement to look at market pricing. So while the overall run value might be able to be neutralized by position, the size of the available talent pool varies.

    When we go looking for a replacement SS, we aren't considering Jesse Gutierrez or Eric Hinske --- even if they might in reality have a greater total run value at SS than Juan Castro. So as we consider what replacement level looks like, we should restrain our description to that population from which we would actually draw a specific replacement.

    Which begs the question, are we artificially limiting our pool of potential players at each position to our detriment? Or rather, is the system really running efficiently with players positioned such as to maximize their overall run production value?

    I think your point in the last paragraph is perhaps the key to the whole series. A run is a run. The goal is simply to accumulate more than the opponent and you can achieve that both by creating them and preventing them.

    ReplyDelete
  13. Rick,

    Most of what you wrote is in complete agreement with what I wrote in the original post under the first explanation for why offense varies by position: there is a smaller talent pool of players that can play the most difficult/unique positions, which means that there's less competition for offensive excellence. Which I argued is almost certainly correct, and must be (a part of) why offense varies across positions. So I think we're in complete agreement that this is an important difference when comparing players across positions.

    Now, clearly, if a player is part of a select group of individuals (e.g. someone who can competently play center field), he is worth more to a team than someone who can only competently play first base, because "everyone" can play first base. The question is, how much more valuable is he?.

    The traditional approach, which I think(?) you're advocating, is to compare individuals strictly to others within their own position. So replacement level at a position is 73% of hitters at that position, and league average fielding at that position. This way, you're only comparing a player against his opposing numbers on other teams, which theoretically sets all players on a level playing field.

    The problem I see with doing that is that measuring players who are currently playing a given position does not give you a complete picture of the pool of talent that could play that position. For example, Troy Tulowitzki would play a badass second base, and would be among the best hitters at that position. But he's excluded from that pool because he's good enough to play another, more demanding position. This can, and demonstrably does, result in talent level differences among positions, like the fact that the "standard" for both hitting and fielding are both lower at second base than center field, according to Tango's data.

    The approach I'm arguing for takes the position that the reason for there being smaller talent pools that can play one position vs. another is that some positions are harder. Therefore, it tries to quantify how much harder, in terms of runs, some positions are vs. the other. The argument is that you could take average-fielding first baseman and stick him at shortstop...but if you did that, what would be the cost that you'd incur on defense? That cost seems to me to be a good estimate of the value of a shortstop over a first baseman. Therefore, player value = [offense vs. lg-replacement] + [defense vs. replacement=average] + [position difficulty adjustment]

    I'm still not seeing any problems with that approach, except that it's hard to quantify position difficulty. In theory, at least, it makes sense to me.

    Thinking about it, there might be a third approach that makes some sense: instead of using offensive averages at a position, use offensive averages among players that could theoretically play that position. We can use the defensive spectrum derived from Tango's data or the Fans' data to do this, which makes it: C-CF-SS-2B-3B-RF-LF-1B-DH.

    Then, for each position, we could quantify the production of those players that qualify for that position. So catchers and probably center fielders would be compared only against one another. First basemen would be compared against everyone except DH's. DH's would be compared to everyone.

    We might even be able to use the Fans Scouting report to further hone down those cutoffs: e.g. only those players with at least decent throwing arms would be eligible for third base, but range isn't so much an issue...

    The question, once we have the pools, would be how to then look for replacement level. We couldn't use the mean, because adding weaker offensive positions to the pool of talent available to poor defensive positions lower that standard--and that's not what we're after. But we could try to do some kind of quality-based sorting (maybe regressed rates or something?), and then look at the 60th or so individuals (e.g. what you get after removing 2 players per team for that position).

    I wonder what those data would look like. I have a feeling that it'd be too much of a penalty for first basemen, but I'm not sure. I might try to do this someday, but it wouldn't be easy.
    -j

    ReplyDelete
  14. I think I am arguing for the hybrid type model you begin to describe at the end. What I don't understand is this assertion:

    "The question, once we have the pools, would be how to then look for replacement level. We couldn't use the mean, because adding weaker offensive positions to the pool of talent available to poor defensive positions lower that standard--and that's not what we're after"

    Why not? Why are we afraid to lower the offensive threshold. It seems to me that that is in fact reality. The fact is that offense is offense. The only reason that we require less offense at SS is because it is difficult to find a player who's defense at SS does not offset his offensive value.

    Thus, in LF, the pool of available offensive talent is enormous precisely because the defensive requirement is so low. You can find a lot more people who provide a .800 OPS and play a good LF than you can who do the same at SS. However, because the standard for entry is so low, it requires somebody of significant offensive value to break in to our magic top 30 (or 60).

    It is precisely this pressure which encourages us to push guys as far up the defensive spectrum as possible.

    I think the real trick is to consider replacement offensive and defensive performance separately and not together. Then you define replacement as that level of performance which places you at the top X players at the position. And unless you put in an artifical floor for defensive ability, then you must consider all position players in the calculus at each position.

    So the offensive replacement value is the same at every position. However, the defensive replacement value swings quite a bit. The problem with Dunn at SS (or Ryan Braun at 3B) is that he's so far below what you can freely get defensively that it negates a great deal of his positive offensive value. The problem with a guy like Neifi Perez is that his crappy offense negates all of his defensive value (and more).

    So basically, I do think you have the right basic equation. The issue is that offensive value is constant regardless of position and defensive value varies by position. This matters once we get in to financial valuation.

    - An aside -
    The last part of my hypothesis is that the defensive spectrum isn't linear at the extremes. That is to say, if you take the best CF and put him at 1B, he's not equally better in terms of raw runs than the average 1B than he is compared to the average CF. In CF, the spread of possible defensive contributions is wider and thus differences between ability more valuable. So a guy who is +30 in CF might only be worth +35 at 1B, not the +43 Tango's model would suggest.

    - End Aside-

    For example let's call Dunn a +80 offensive guy and a -25 position neutral defensive guy. At SS he's worth just +51 where as in LF he's worth +59. If the linear model works, you can just accumulate the most valuable position-neutral guys you can get your hands on and it won't matter where you stick them defensively. But this strikes me (perhaps incorrectly) as intuitively wrong.

    However, what if Dunn (-25) isn't just a -21 in LF and -29 at SS, but is actually -40 at SS. Also, what if our SS (+5) isn't just +1 at SS and +9 in LF, but is only +6 in LF. I fear that this average technique is flattening out a non-linear spectrum and thus minizing the true effect of good defense at the "tough positions", underestimating the increased influence of moving left in the spectrum and overestimating the influence of moving right.

    For example. Pujols is +30 at 1B (+22 pos neutral) and Ichiro is +33 in CF (+38 pos) neutral. I don't think it's likely that Pujols would be +17 in CF (I think he'd be lower) than Ichrio would be +46 at 1B (I think he'd be lower). Perhaps it's as simple as the fact that the skill sets difference we've touched on comes in to play.

    Where players have moved positions it's been in those cases where they happened to have skills which translated. However, it's possible that these players are not representative of the general sets of abilities held by their position mates. So while Chone Figgins, Pujols, or Soriano can (and have) switched positions, that was largely due to their possession of two only partially related skill sets. Scott Rolen and Juan Pierre may possess both sets or at least at the level of those who have switched.

    In general, moves to the left on the spectrum would generally result in bigger losses and moves to the right result in smaller gains.

    Which leads me to hypothesize that the defense conversion scale cannot have two way arrows between each stop but needs to have a different scale for moving in either direction. Using a defense neutral measurement based on a methodology like Tango used incorrectly flattens out these curves due to a biased sample predisposed to such adaptability leaving a general difficulty measurement that cannot be generally applied.

    ReplyDelete
  15. Rick,

    1. The reason that you can't use the mean in my example at first base is that it would result in a lower standard (and thus a greater boost) for first basemen than we're already using with offensive disparities. Right now 1B's are compared against 119% of league average. Using the mean of all NL players would put them at just about 100% of league average, which would mean that 1B's would be considered more valuable than ever (if anything, they should be less valuable, because there is more competition at that position than any other defensive position).

    What we're interested is, as you said, an indication of who the top X possible players are at their position. Which is why, if using one year of data, I'd probably try to construct some sort of regressed rate-based ranking within each position's theoretical pool, remove the top 60 players or so (1 starter + 1 bench), and then look at the next set of players to find replacement level. Might be neat to try at some point.

    2. I think it's an error to consider offense and defense separately for each player. A very strong defensive player can be an above average player despite bad offense (e.g. Adam Everett). And vice versa (e.g. Hanley Ramirez, Derek Jeter). You can't just think about it as what "defense" can you freely get, because that defense is always going to come coupled with offense in your replacement player. You have to think about as what player could you freely find, and what his total value would be (or, what his typical offense and typical defense will be, which is the approach I took in my replacement study).

    Tom Tango often says that there's no such thing as replacement-level offense or replacement-level defense. Instead, there are replacement players, and that's what we're after. I think he's right. That's why, if we can get an accurate depiction of position value, it probably would be best to scrounge up as many top position-neutral value players as you could (regardless of position), and then stick them into positions as best you can to maximize the leverage of each players' defensive skills. It might mean that some players will play out of their "natural" position, but that's certainly something that happens in real life...and at some point, the [runs scored] + [runs allowed] makes that a defensive or offensive loss worthwhile.

    3. I expect you're right that the defensive spectrum is both oversimplified and non-linear, probably in a variety of ways. It is the case that players generally move down the defensive spectrum, and not up it, so it might be hard to test your directionality idea with a good, minimally biased sample in both directions.

    One thing that might be tractable would be to down-weight extreme values (log-scale?) within each position and then re-do Tango's study. Thinking about it, though, I wonder if that would just make the positional disparity estimates even more conservative, since it's the extreme guys who are (probably) likely to have the bigger-than-usual changes.

    The goal, to me, is to get the best estimate I can of player value. It's always going to be an estimate, as we can never control or account for everything. But I still think that we can get a pretty good idea using the approach I'm advocating. And as better estimates of position value disparities come out, I'll gladly adopt those new values in my work.
    -j

    ReplyDelete
  16. Justin, so what are you using as your positional multipliers?

    Your line in one of these comments that "replacement level fielding IS league-average fielding" resonated with me. I always thought of it as "replacement-level players are average fielders and crappy hitters", but having replacement level players be both replacement level hitters and fielders sits better. Nice.

    To those who don't like the fact that different players would perform differently at different positions, it doesn't really matter for what Justin's trying to measure, which is value. Adam Dunn actually played LF. Even if he was a slick fielder and would be a great SS, that's not where he played. He was 20 runs worse than the average LF, who's 10 runs less valuable than the typical shortstop. Value comes from out-performing the alternative and the skills of the typical alternative don't depend on Dunn's actual ability as a SS.

    If you're not trying to measure value -- say, you want to know how much Adam Dunn would help your team as a shortstop if you signed him as a free agent -- then yes, you definitely want to know how Adam Dunn will play shortstop. Scouting is important. Tango's Fan Scouting Report says that his instincts, first step, and hands are dreadful, likely making him a worse SS than LF.

    ReplyDelete
  17. Hey Sky,

    Do you mean position multipliers on the offensive numbers? I'm not.

    I'm just calculating offense vs. league-wide replacement hitting (which I'm defining as 73% of league average).

    I'm then adding in fielding vs. average at a position. And then adding in a pro-rated (by defensive innings) position adjustment for runs (I'm using Tango's numbers). If players play multiple positions, I pro-rate the position adjustment for each position and then sum 'em up. :)
    -j

    ReplyDelete
  18. How are you pro-rating the innings? If a CF is +10 over a full season, are you defining full season as 162*9 innings? Per 150 games? Something else? How are you handling DH "innings"?

    ReplyDelete
  19. I was sort of lazy and just used Chris Dial's estimate of ~1440 defensive innings per team per year as the denominator, and player innings at a position as the numerator. I should probably go and check that number to make sure it's a reasonable estimate these days, but I just trusted him...and it seems to work.

    As for DH's....eek. I fortunately am mostly interested in National League (namely, the Reds), so I haven't had to deal with that. I don't have a great solution either. Maybe if you have games started as a DH, you can then just estimate 9 innings per game started to get at least some idea. DH's are so problematic anyway, since it's hard to know what to use for a position adjustment for them... -j

    ReplyDelete
  20. It is important to keep repeating that there is no replacement level hitting or replacement level fielding. Just replacement level players. You can say hitting level of replacement level players, and fielding level of replacement level players. And, as has been shown on my site, the fielding level of those players is very close to the average for their position.

    ReplyDelete
  21. Hi Tango,

    Thanks for your post in my series--I like the phrase "hitting or fielding level of replacement players." Nice way to put it.

    I also recently did a replacement study, and found much the same results as you did with respect to fielding. -j

    ReplyDelete
  22. The skills that are important at some positions correlate better with hitting well than others. So a 2B might be absolutely worse than a CF, but it's more unusual to find 2B-type skills in a competent hitter than CF-type skills, because CF-skills correlate better with hitting than 2B skills.

    The point is that it's not just about how absolutely difficult a particular position is to play to a major-league competent level; it's also how good the pool of hitters is that can do that, ie the pool of CFs isn't that big, but it's deep; the pool of 2Bs might be bigger, but it's shallower.

    ReplyDelete