Table of Contents

Saturday, April 19, 2008

Friday Night Fungoes: Dan Fox, Projections, and the Future, man!

Pirates sign Dan Fox

The Pirates continue to their overhaul of their front office by hiring Dan Fox, aka Dan Agonistes, who has written at both Baseball Prospectus and The Hardball Times. His official title is Director of Baseball Systems Development, so I am not sure if this is the same job that Tango advertised a while back--but it might be. The responsibilities sound similar--responsible for "integrating the array of quantitative and qualitative information in a way that makes both even more instructive."

Obviously I'm thrilled for Dan. This no doubt is incredibly exciting for him, and I'm sure the Pirates made a heck of a hire to get him. But this is kind of a bummer for me in a few ways:

1. The Pirates as an organization probably just got a bit better. As a Reds fan, that's not a good thing.
2. It means that we as a community lose access to an exceptionally good, broadly trained, and very resourceful analyst.
3. Dan Fox is one of the three main reasons that I decided to continue to subscribe to Baseball Prospectus this year. So now I'm down to PECOTA and Kevin Goldstein's work on prospects, though I'll give a nod to Nate Silver's occasional high-quality article as a secondary reason.

I have to say, I might start paying a little bit more attention to the Pirates in coming years. I like what I see and hear from their front office since they hired Neal Huntington, and I will be living minutes from their AA-franchise. I don't know if I could ever really leave the Reds and root for an NL Central foe, but you never know.

Projection System Showdown

I missed this when it happened, but (with a hat tip to studes) Tom Tango recently did a rather careful study of projection accuracy for hitters in the 2007 season.

Before I get into Tango's study, I do want to point out that his results are similar to those reported by Nate Silver last year (hitters & pitchers here). There are a few subtle methodological differences, though, and Silver focused more on rank order of the systems than the actual impact of any differences seen. The latter is probably the most important point, as you'll see...

Anyway, Tango followed two basic steps:

1. Adjust forecasted league-average OPS to match actual 2007 league-average OPS.
2. Determine the average deviation between expected OPS value and actual OPS value (i.e. the average residual).

He reported overall average error, as well as four groups based on total multi-year MLB plate appearance totals: high (grizzled veteran regulars), medium, low, and rookies (made debut in 2007). The results can be hard to sift through in that very long thread (starts on #29, ends on #103), so I opted to whip up a quick graph (hope Tango doesn't mind). Note: I inverted the y-axis to read more intuitively--the "higher" the points are (vertically), the better, because they show less error.You may want to open the graph in another tab to view it. I tried to stretch it to separate the lines as much as I could, within reason. Here are the primary conclusions of Tango's study:
  • There isn't much difference between the different projection systems.
    • Overall, the best system (PECOTA) provided OPS measurements that were, on average, 0.069 off from the actual values. That means that a player projected to have an 0.800 OPS will, on average, have an OPS between 0.731 and 0.869. Not particularly accurate, really.
    • The worst system, aside from just projecting everyone to be league-average, had an average error of 0.075 OPS points. So, again, an 0.800 OPS player will, on average, have an OPS between 0.725 and 0.875. Not particularly worse than the "good" system.
    • Free systems CHONE and ZiPS were a whopping 0.001 OPS units "worse" than the much-lauded PECOTA. Marcel was only 0.002 OPS units behind. And they beat for-cost systems like Bill James' and Shandler's.
    • The best thing to do is to take an average across all nine projection systems. But even then, you only get an extra "point" of OPS accuracy over PECOTA and 3 points over Marcel. Again, BFD.
    • Despite all the additional information that systems like CHONE and PECOTA have about minor league players, they're only marginally better than Marcel when projecting players with few MLB plate appearances.... and in those cases, Marcel can only projects league average.
  • Forecasting playing time is hard.
    • The Fans' community forecast didn't do better (or worse) than the objective systems in projecting OPS. However, they DID do substantially better when projecting playing time.
    • Sal Baxamusa and Tango think that a system combining the Fans' projections of playing time and Marcel's simple projections of performance would probably outcompete anything out there in a head-to-head contest.
Overall, in terms of accuracy, we can do just as well--if not better--using CHONE, ZiPS, or Marcel (all free) as we can using the for-cost PECOTA, Bill James, or Shandler projections. I think I knew this was true in theory, but always figured I was still gaining something by using PECOTA over something like Marcel. Now, I'm feeling doubtful...

Two caveats to this study: 1) it only looks at hitter projections, and (most importantly) 2) it only looks at 2007 data.

Even so, I guess I'm not seeing a compelling reason to use PECOTA at this point. So, on my list of reasons to renew my BPro membership, I guess I'm down to Kevin Goldstein. ... I enjoy Kevin's column, but I'm not sure if that'd be enough if I were renewing today.

The Future of Sabermetrics, man!

There were two Hardball Times articles this week that I thought did a tremendous job of setting the stage for a lot of the work we're likely to see in the coming years:
  • Sal Baxamusa did a great job of putting forth an organizational scheme to help us understand baseball player valuation. I find those sorts of charts to be tremendously effective ways to organize my thoughts. The first chart he presented is basically what I worked through in my player value series last winter. What Sal does is take it the next step and show how we might better understand pitching, especially in light of the influx of pitchf/x data. We could do similar charts for hitting, baserunning, and fielding--all of which might be influenced by the "f/x" line of data (hitf/x, fieldf/x, etc).
  • Mike Fast put forth a litany of questions that remain to be answered in baseball research. What makes his list useful is that all of these may be answered, at least in part, via the use of ball tracking data. Therefore, we may see tremendous progress on most of these within the coming year or two. Exciting times are ahead...

Signing Young Players Early

Skyking has a great post breaking down the contract extension of Evan Longoria with the Rays. He finds, as did Tango, that Longoria is almost certain sacrificing a huge amount of money by signing this deal today. On the other hand, he just guaranteed that he'll make $17.5 million, even if he goes and has a career-ending injury tomorrow.

As Sky points out, that first $17.5 million would probably worth much more to most of us then the $50 million that might come after it. He made a similar point after the Granderson signing. I'm sure that I'd have a hard time not taking the $17 million today, even if I knew I'd probably make three times that if I went year-to-year, simply because the risk could be so catastrophic.

I would not be surprised to see the Reds try to work out similar deals with Joey Votto and (especially) Jay Bruce this season or offseason. In fact, I expect them to do it: it makes too much sense NOT to do it. I'm not sure about Cueto, Volquez, and Bailey, though...pitchers are so darn injury prone that it's pretty dangerous to sign them long-term. Still, the kinds of discounts that teams are getting when signing players like Granderson, Tulowitzki, and now Longoria are so extreme that at some point it will make sense to lock up young pitchers as well.

More MLB customer service issues: Blackouts

Last week, I mentioned a series of recent issues related to MLB's tendency to not necessarily keep the best interests of their fans in mind. I forgot a big one: blackouts. Maury Brown posted a BPro article (I guess that's a reason to keep my subscription) about TV blackouts last weekend that is worth a read, though I frankly still just don't understand what MLB is thinking. Here's my favorite quote from it:
Only in baseball would there be a collective head nod to the idea that it's good business practice to restrict consumers' access to your product.

Also, I just became aware of a blog that has been started up in protest of MLB's blackout policy. This will be a good way to keep tabs on the issue. FWIW, they report that there may be some tangible effort to at least remove the most absurd blackout restrictions in the near future.

Reds Shut Down Scout's Blog

Finally, this week saw the birth and death of a blog by one of the Reds' scouts, Butch Baccala. He apparently was shut down due to concerns about giving out information that might put the Reds at a competitive disadvantage. To my eye Butch clearly knew where that line was and would not cross it. The Reds apparently disagreed.

I see this as yet another example of the Reds' penchant for absurd secrecy. And frankly, I find that both annoying and disappointing.

Dave from Louisville thinks I feel this way because I'm an academic. Thank goodness I'm never going to have to work in the corporate world, because it sounds awful.