The Pirates continue to their overhaul of their front office by hiring Dan Fox, aka Dan Agonistes, who has written at both Baseball Prospectus and The Hardball Times. His official title is Director of Baseball Systems Development, so I am not sure if this is the same job that Tango advertised a while back--but it might be. The responsibilities sound similar--responsible for "integrating the array of quantitative and qualitative information in a way that makes both even more instructive."
Obviously I'm thrilled for Dan. This no doubt is incredibly exciting for him, and I'm sure the Pirates made a heck of a hire to get him. But this is kind of a bummer for me in a few ways:
1. The Pirates as an organization probably just got a bit better. As a Reds fan, that's not a good thing.
2. It means that we as a community lose access to an exceptionally good, broadly trained, and very resourceful analyst.
3. Dan Fox is one of the three main reasons that I decided to continue to subscribe to Baseball Prospectus this year. So now I'm down to PECOTA and Kevin Goldstein's work on prospects, though I'll give a nod to Nate Silver's occasional high-quality article as a secondary reason.
I have to say, I might start paying a little bit more attention to the Pirates in coming years. I like what I see and hear from their front office since they hired Neal Huntington, and I will be living minutes from their AA-franchise. I don't know if I could ever really leave the Reds and root for an NL Central foe, but you never know.
Projection System Showdown
I missed this when it happened, but (with a hat tip to studes) Tom Tango recently did a rather careful study of projection accuracy for hitters in the 2007 season.
Before I get into Tango's study, I do want to point out that his results are similar to those reported by Nate Silver last year (hitters & pitchers here). There are a few subtle methodological differences, though, and Silver focused more on rank order of the systems than the actual impact of any differences seen. The latter is probably the most important point, as you'll see...
Anyway, Tango followed two basic steps:
1. Adjust forecasted league-average OPS to match actual 2007 league-average OPS.
2. Determine the average deviation between expected OPS value and actual OPS value (i.e. the average residual).
He reported overall average error, as well as four groups based on total multi-year MLB plate appearance totals: high (grizzled veteran regulars), medium, low, and rookies (made debut in 2007). The results can be hard to sift through in that very long thread (starts on #29, ends on #103), so I opted to whip up a quick graph (hope Tango doesn't mind). Note: I inverted the y-axis to read more intuitively--the "higher" the points are (vertically), the better, because they show less error.
You may want to open the graph in another tab to view it. I tried to stretch it to separate the lines as much as I could, within reason. Here are the primary conclusions of Tango's study:- There isn't much difference between the different projection systems.
- Overall, the best system (PECOTA) provided OPS measurements that were, on average, 0.069 off from the actual values. That means that a player projected to have an 0.800 OPS will, on average, have an OPS between 0.731 and 0.869. Not particularly accurate, really.
- The worst system, aside from just projecting everyone to be league-average, had an average error of 0.075 OPS points. So, again, an 0.800 OPS player will, on average, have an OPS between 0.725 and 0.875. Not particularly worse than the "good" system.
- Free systems CHONE and ZiPS were a whopping 0.001 OPS units "worse" than the much-lauded PECOTA. Marcel was only 0.002 OPS units behind. And they beat for-cost systems like Bill James' and Shandler's.
- The best thing to do is to take an average across all nine projection systems. But even then, you only get an extra "point" of OPS accuracy over PECOTA and 3 points over Marcel. Again, BFD.
- Despite all the additional information that systems like CHONE and PECOTA have about minor league players, they're only marginally better than Marcel when projecting players with few MLB plate appearances.... and in those cases, Marcel can only projects league average.
- Forecasting playing time is hard.
- The Fans' community forecast didn't do better (or worse) than the objective systems in projecting OPS. However, they DID do substantially better when projecting playing time.
- Sal Baxamusa and Tango think that a system combining the Fans' projections of playing time and Marcel's simple projections of performance would probably outcompete anything out there in a head-to-head contest.
Two caveats to this study: 1) it only looks at hitter projections, and (most importantly) 2) it only looks at 2007 data.
Even so, I guess I'm not seeing a compelling reason to use PECOTA at this point. So, on my list of reasons to renew my BPro membership, I guess I'm down to Kevin Goldstein. ... I enjoy Kevin's column, but I'm not sure if that'd be enough if I were renewing today.
The Future of Sabermetrics, man!
There were two Hardball Times articles this week that I thought did a tremendous job of setting the stage for a lot of the work we're likely to see in the coming years:
- Sal Baxamusa did a great job of putting forth an organizational scheme to help us understand baseball player valuation. I find those sorts of charts to be tremendously effective ways to organize my thoughts. The first chart he presented is basically what I worked through in my player value series last winter. What Sal does is take it the next step and show how we might better understand pitching, especially in light of the influx of pitchf/x data. We could do similar charts for hitting, baserunning, and fielding--all of which might be influenced by the "f/x" line of data (hitf/x, fieldf/x, etc).
- Mike Fast put forth a litany of questions that remain to be answered in baseball research. What makes his list useful is that all of these may be answered, at least in part, via the use of ball tracking data. Therefore, we may see tremendous progress on most of these within the coming year or two. Exciting times are ahead...
Signing Young Players Early
Skyking has a great post breaking down the contract extension of Evan Longoria with the Rays. He finds, as did Tango, that Longoria is almost certain sacrificing a huge amount of money by signing this deal today. On the other hand, he just guaranteed that he'll make $17.5 million, even if he goes and has a career-ending injury tomorrow.
As Sky points out, that first $17.5 million would probably worth much more to most of us then the $50 million that might come after it. He made a similar point after the Granderson signing. I'm sure that I'd have a hard time not taking the $17 million today, even if I knew I'd probably make three times that if I went year-to-year, simply because the risk could be so catastrophic.
I would not be surprised to see the Reds try to work out similar deals with Joey Votto and (especially) Jay Bruce this season or offseason. In fact, I expect them to do it: it makes too much sense NOT to do it. I'm not sure about Cueto, Volquez, and Bailey, though...pitchers are so darn injury prone that it's pretty dangerous to sign them long-term. Still, the kinds of discounts that teams are getting when signing players like Granderson, Tulowitzki, and now Longoria are so extreme that at some point it will make sense to lock up young pitchers as well.
More MLB customer service issues: Blackouts
Last week, I mentioned a series of recent issues related to MLB's tendency to not necessarily keep the best interests of their fans in mind. I forgot a big one: blackouts. Maury Brown posted a BPro article (I guess that's a reason to keep my subscription) about TV blackouts last weekend that is worth a read, though I frankly still just don't understand what MLB is thinking. Here's my favorite quote from it:
Only in baseball would there be a collective head nod to the idea that it's good business practice to restrict consumers' access to your product.Awesome.
Also, I just became aware of a blog that has been started up in protest of MLB's blackout policy. This will be a good way to keep tabs on the issue. FWIW, they report that there may be some tangible effort to at least remove the most absurd blackout restrictions in the near future.
Reds Shut Down Scout's Blog
Finally, this week saw the birth and death of a blog by one of the Reds' scouts, Butch Baccala. He apparently was shut down due to concerns about giving out information that might put the Reds at a competitive disadvantage. To my eye Butch clearly knew where that line was and would not cross it. The Reds apparently disagreed.
I see this as yet another example of the Reds' penchant for absurd secrecy. And frankly, I find that both annoying and disappointing.
Dave from Louisville thinks I feel this way because I'm an academic. Thank goodness I'm never going to have to work in the corporate world, because it sounds awful.
That's interesting...I was recently hired as a scout for the Cubs (yeah, lifetime Reds fan) and I have also have a site. I've been writing about some similar things...more day to day stuff than anything, and some bits on players already signed. I would never write about a player I was following, but neither did he.
ReplyDeleteMy boss reads my writing. In fact, I think it's part of the reason I was hired.
Also, it's surprising to read scouting and "corporate world." I never thought about it that way...mostly because "the office' is a ballfield for the most part.
I think you may be surprised how corporate academia can be. I suppose it depends on what kind of academia you're talking about.
That said, I wish you all the luck. Don't let it take too much time away from this great site.
I'm REALLY excited to see what the Pirates do in the next five years. I fully expect them to be the next Indians/Diamondbacks.
ReplyDeleteThe signing of young players to long-term contracts will only continue to proliferate, until there's an uprising against have to sacrifice so much future money just to acquire financial security. My guess is that the next big labor disagreement will be about those first six (and especially the first three) team-controlled seasons.
(By the way, there's a typo in the second Longoria paragraph. "Less" should be "more". Thanks for the link.)
My point is that working in an open peer review environment is the direct opposite of how a business needs to operate in order to be a step ahead of the competition.
ReplyDeleteIt's not awful at all, its just the way it is. Most jobs have have a given level of confidentiality; physicians, therapists, nurses, policeman, lawyers, anyone trying to get a patent, hedge fund managers, and military workers can't discuss in depth their jobs.
Any kind of bureaucracy is going to have similarities. I believe you'll find that education is just as bad as a corporation.
Love the blog and hope you keep it up when you start working. That being said...............Why would you start cheering for the another team just when we start getting good?????
Good luck with the move, as will be a pain in the ass.
Justin -
ReplyDeleteOutstanding work, as always.
I responded to yours and Dave's comments on Butch's blog. I think that you each have valid points but I side with Dave.
Thanks for the head's up on the MLB Blackout blog. In the past, I downloaded the MLB blackout database and informed MLB that I lived in some backwater in southern California after building an analysis model to figure out which zip code would allow me to be blacked out of the fewest Reds' games. I got away with it for a year, but in the end I was contacted and "scolded" for my deception. It's tough to live in central Arkansas and be blacked out of eighteen games a year because the Reds are in the same division as the Astros and Cardinals, each 300+ miles away.
I applied for the job that Dan Fox took. It's hard for me to think of a more qualified candidate than him, even though it hurts my ego a bit to admit that. I don't know that the Pirates are going to turn things around until their ownership group starts making better decisions, but having Dan around can do nothing but help.
Cheers,
Chris
Regarding academic jobs:
ReplyDelete1. I have a pretty good idea of how these positions work--after all, I've been immersed and closely interfacing with faculty for, what, 9 years (MS + PhD)? Graduate school in the sciences involves a very close relationship with a faculty mentor (and often other committee members), which I know is different from other fields. The only "secrecy" that exists, at least in the track that I'm pursuing, is that one doesn't discuss a students' grades with anyone who doesn't need to know. But that's a privacy issue, not an issue of competition. I'm pretty big on privacy--that's part of the reason I don't use my real name! :) The openness of this field was a major attraction for me when I started down this road over a decade ago.
2. One of the reasons that academic research is so effective is that it is so open. The more that people share information, and the more heads thinking about and hands working towards the same end goal, the better. It's also why, I believe, that we see some of the best research in baseball going on at places like The Hardball Times and The Book Blog (and smaller websites), and why Baseball Prospectus is in many ways falling behind the times. I'm sure the teams are doing pretty well given the resources they have to throw at research, but there's a reason that people like Dan Fox, Joe P. Sheehan, etc, keep getting hired: they're some of the best in the world at what they do.
3. The first year in my new position might be a bear as I get my lectures set up. After that, life's going to settle down a lot. Even so, this current semester is the hardest I've ever had--writing dissertation, TAing, and (especially) teaching a night class three nights a week. It's been killing me, and the blog is honestly about all that keeps me sane at times. So if I can keep it up this semester, I'm sure I'll be able to keep it up with the new job. :)
@Aaron, I'd love to get a link to your blog, either here or via e-mail. :)
@Sky, thanks for the typo catch. I agree that this would seem to be the next big thing in bargaining. It's going to be tough to change though--if you triple the minimum salary, for example, that's going to really hurt small market teams...
@Chris, I think the Pirates' ownership most important decision was the hiring of Neal Huntington. It was a clear change in direction, and since then they've been making progress. Like Sky says, it might take a good five years to get on the right path--their minor league system is still pretty barren. But I've been pretty impressed with them thus far.
-j
Thanks for pointing the blackout issues, Justin. On the article, and the mentions on other outlets regarding adjustments to the blackout policy... The adjustments will be coming at the local blackout level. In other words, if say, there is an area of overlap in Oregon (the bottom three counties where the Giants, A's, and Mariners overlap), it will be a "use it or lose it" deal. In that instance, it's possible that the Mariners will get the territory all to themselves. So, this removes some of the blackout issues, but not all. It will depend on whether the fan impacted in an A's, Giants, or Mariners fan... Lastly, don't look for anything at the national level to be changed. Those deals are locked in. Nice that we're all sitting here on the weekend looking at blank screens. For those that don't have a BP subscription, I wrote much of what was in the BP article here: Back in Black: Extra Innings and MLB.TV Blackouts
ReplyDeleteIn the world of competitive intelligence, it's all about connecting dots that are often fuzzy & far apart. And sometimes you need to connect the dots that aren't there.
ReplyDeleteIt is sorta like the spook world - you may save the country, but your medal never leaves CIA or NSA hq. It's got to be tough on the scouts - read the book by the old scout from Fostoria, OH, Prophet of the Sandlots (I think).
But there is a positive - that this info IS important, whether sabermetric or scouting. If they don't want it shared, it's seen as having value.
Good recap.
ReplyDeleteI also do the pitchers on my blog:
http://www.insidethebook.com/ee/index.php/site/article/community_forecast_2007_pitcher_results/
Thanks--and I missed the pitchers! I guess I just wasn't paying attention the community forecast for some reason, and overlooked the analysis in each of those posts.
ReplyDeleteCHONE's looking like the route that I'll go moving forward. But you can't really go wrong using any of the free systems (marcel, zips, or chone). -j