Table of Contents

Sunday, April 13, 2008

Markov: Dusty Baker's lineups aren't half bad

Update: Due to several errors with how I was using John Beamer's Markov model, this study is frankly a load of hooey. I am leaving it here for archival purposes, but please disregard pretty much everything here. Sorry about that--I had done several checks to be sure I was "doing it right," but it turns out that the specific ways I was thinking about the input were absolutely incorrect. Embarassing to say the least, but that's what happens with research now and then...

I'll post an updated version of this study when I get a chance. The results are far less surprising, and much more in line with other work on lineups than these were.


There was a chorus of complaints from Reds' faithful over the Reds' opening-day lineup, and this has continued in reaction to subsequent lineups over the Reds' first two weeks of play. I asked folks to submit to me what they felt the opening day lineup should have been in a "Can you make a better lineup than Dusty Baker" contest of sorts.

I've now taken those lineups and, with the help of PECOTA '08 projections, I've plugged them all into John Beamer's Markov Chain spreadsheet that came with the Hardball Times 2008 Annual. The results surprised me, and I expect that they'll surprise a good number of you as well.

Background - Evaluating Lineups (you can skip if you want)

The past several years have seen increasingly sophisticated work done in the area of lineup construction. Among the most widely-publicized tools that stems from this effort is David Pinto's lineup tool, which I first discussed roughly two years ago. It is based on a set of studies that used linear regressions to relate player OBP and SLG at each lineup position to runs scored. Those regressions indicated that we should make some changes to how lineups have traditionally been designed. Around the same time, The Book was published; it used a different approach, but arrived at many of the same conclusions. Check out my old post for details.

The problem with approach upon which Pinto's tool, in particular, is built is that a lineup is an incredibly dynamic thing. Regressions just report typical relationships between OBP and SLG across different lineup spots based on how MLB managers have filled out their lineup cards in the past. That's a different thing from having a tool that you can use to try out radically different lineups, as Pinto's tool permits one to do. For example, if you hit the pitcher 1st, that will have large effect on the run-producing opportunities of the #2 and #3 hitters compared to hitting an on-base machine like Scott Hatteberg 1st. In other words, the player placed into each of the lineup slots will have direct effects on the opportunities of all other players in the lineup, and the consequences of a seemingly minor substitution might not be immediately apparent. Regression simply will not capture these interactions.

Enter Markov Chains. Markov chain models are models that organize complicated processes into steps. At each step, you input a certain probability that a variety of specific events will occur. The result is a series of branching event chains, which are then summarized according to their probabilities. John Beamer described them here in his introduction to his model.

It turns out that this is a great way to model a lineup's performance--if you can input the chances of different offensive events happening for each player (using, for example, PECOTA projections), you can have Markov step through a lineup throughout a game, keeping track of outs and innings, all the while keeping track of the entire range of possibilities in terms of offensive production. In other words, if you can design a Markov model that fits the game of baseball, and give it accurate data, it will provide you with more precise estimates of offensive production by a lineup than with the regression coefficients can ever hope to do because it actually attempts to simulate baseball.

In the 2008 THT Annual, John Beamer released an excel-based Markov chain model that is apparently capable of accurately modeling baseball just as I described above. It is probably the best publically-available tool for lineup analysis.

2008 Opening Day

First, let's look at the Opening Day lineup, isolating our choices to only those players that Dusty chose to play (i.e. no Joey Votto, no Jay Bruce, etc). That way, we can focus more on the effect of moving players around the batting order, and less on player substitutions.

Below is a table of the Reds' opening day players, along with their 2008 PECOTA projections broken down into my favorite set of diagnostic stats:

Last First PA %K %BB BABIP AVG OBP SLG ISO OPS lwts_RC R/G wOBA
Patterson Corey 458 16% 5% 0.306 0.268 0.307 0.402 0.134 0.709 56.3 4.76 0.300
Keppinger Jeff 514 6% 8% 0.317 0.305 0.364 0.418 0.113 0.781 70.6 5.82 0.336
Griffey Ken 435 16% 11% 0.283 0.268 0.350 0.480 0.213 0.830 65.7 6.20 0.354
Phillips Brandon 629 16% 6% 0.302 0.274 0.325 0.444 0.170 0.769 86.4 5.43 0.324
Dunn Adam 579 25% 16% 0.298 0.261 0.388 0.549 0.288 0.937 103.0 7.73 0.392
Encarnacion Edwin 561 16% 8% 0.308 0.285 0.356 0.493 0.208 0.850 88.5 6.51 0.354
Hatteberg Scott 278 9% 11% 0.296 0.285 0.368 0.440 0.155 0.808 40.3 6.12 0.352
Valentin Javier 208 12% 9% 0.287 0.269 0.333 0.424 0.155 0.757 27.1 5.22 0.327
Pitcher Pitcher 363 35% 3% 0.202 0.123 0.156 0.153 0.030 0.309 ---
---
---
Note: Markov pays no attention to plate appearances, except to generate frequencies of each even happening. Therefore, it doesn't matter if a player has 200 or 500 PA's, what matters are the number of singles, doubles, strikeouts, walks, etc, per plate appearance. Also, Markov doesn't know about lefty/right splits--it just assumes an average pitcher, who is mostly (but not entirely) right-handed. :)

I received 14 different lineups from users both here and at RedsZone, and I threw in a few of mine own as well. There are actually 9! = 362,880 possible lineup combinations of these nine players (counting the pitcher slot) that we could theoretically try, but I think these represent a good part of the diversity of recommendations that most folks might like to try. Below, I list all of those lineups, as well as the Markov-based estimate of runs per game that those lineups could be expected to provide. The +/- Baker column lists the season-level differences in performance of each lineup compared to Dusty Baker's true opening day lineup (in italics).

Name 1st 2nd 3rd 4th 5th 6th 7th 8th 9th R/G R/162G vBaker
Bluzer-OD Keppinger Hatteberg Phillips Dunn Griffey Encarnacion Patterson Valentin Pitcher 4.90 793.6 4.5
Chris-OD2 Keppinger Encarnacion Dunn Phillips Griffey Hatteberg Patterson Valentin Pitcher 4.89 792.9 3.8
Pickoff-OD Keppinger Encarnacion Dunn Griffey Phillips Hatteberg Patterson Valentin Pitcher 4.88 790.5 1.4
Baker-OD
Patterson Keppinger Griffey Phillips Dunn Encarnacion Hatteberg Valentin Pitcher 4.87 789.1 0.0
AVG-rank Keppinger Encarnacion Hatteberg Phillips Patterson Griffey Valentin Dunn Pitcher 4.86 788.1 -1.0
brad-OD1 Keppinger Hatteberg Griffey Dunn Encarnacion Phillips Patterson Valentin Pitcher 4.86 787.6 -1.5
SLG-rank Dunn Encarnacion Griffey Phillips Hatteberg Patterson Valentin Keppinger Pitcher 4.86 786.8 -2.3
OPS-rank Dunn Encarnacion Griffey Hatteberg Keppinger Phillips Valentin Patterson Pitcher 4.84 783.4 -5.7
OBP-rank Dunn Keppinger Hatteberg Encarnacion Griffey Valentin Phillips Patterson Pitcher 4.83 782.7 -6.4
texasdave-OD Keppinger Hatteberg Dunn Encarnacion Griffey Phillips Valentin Patterson Pitcher 4.80 777.7 -11.4
Chris-OD Keppinger Hatteberg Encarnacion Dunn Griffey Valentin Phillips Patterson Pitcher 4.79 775.7 -13.4
jinaz-OD Encarnacion Dunn Keppinger Griffey Hatteberg Phillips Valentin Patterson Pitcher 4.77 772.2 -16.9
jinaz-OD-exploit Encarnacion Dunn Hatteberg Griffey Keppinger Phillips Valentin Patterson Pitcher 4.76 771.5 -17.6
Trace's Daddy-OD Hatteberg Dunn Griffey Phillips Encarnacion Keppinger Valentin Patterson Pitcher 4.76 770.7 -18.4
justincredible-OD Hatteberg Keppinger Griffey Dunn Phillips Encarnacion Patterson Valentin Pitcher 4.75 770.0 -19.1
OesterPoster-OD Hatteberg Keppinger Griffey Dunn Phillips Patterson Encarnacion Valentin Pitcher 4.74 768.0 -21.1
mlbfan30-OD Hatteberg Keppinger Dunn Encarnacion Griffey Phillips Patterson Valentin Pitcher 4.72 764.8 -24.3
Degenerate-OD Hatteberg Keppinger Dunn Encarnacion Griffey Phillips Patterson Valentin Pitcher 4.72 764.8 -24.3
fareast-OD Hatteberg Keppinger Phillips Griffey Dunn Encarnacion Valentin Patterson Pitcher 4.70 761.4 -27.7
brad-OD2 Hatteberg Dunn Encarnacion Griffey Phillips Keppinger Patterson Valentin Pitcher 4.69 760.6 -28.5
joel-OD1 Hatteberg Dunn Encarnacion Griffey Phillips Patterson Valentin Pitcher Keppinger 4.68 758.9 -30.2
redsmanrick-OD Hatteberg Dunn Encarnacion Griffey Keppinger Phillips Patterson Valentin Pitcher 4.66 755.5 -33.6

All I can say is "wow." Dusty's lineup didn't come out as #1, but it was darn hard to beat, and only an estimated 5 runs per season behind Bluzer's top-rated lineup.

Think about that. Baker's lineup violates one of the biggest "rules" for lineup construction that us stat people harp on--his leadoff hitter is projected to have a miserable 0.307 OBP this season. And yet, the interactions between players in his lineup are such that his lineup results in more wins per season than most other variants...at least, according to Markov. My own lineups, which I designed based largely on the lineup chapter in The Book, rated as a fairly middle-of-the-pack lineup, and came out a good 17 runs (~1.5 wins) behind Baker's model. And some of the user-submitted lineups, which look very reasonable to my eye, came out more than 30 runs per season behind Baker's. Again, "wow."

A few other observations and interpretations:
  • The range of performances of different lineups was about 38 runs per season, despite all lineups featuring the exact same players. That's almost 4 wins worth of variation! More than I expected to see.
  • Lineups with Keppinger leading off did better than lineups with Hatteberg leading off, despite them being rather similar hitters according to PECOTA--both are high OBP guys, though Hatteberg projects to have more power.
  • Most of the "best" lineups have Phillips batting in the 5th spot or higher in the lineup. Many of the "worst" lineups (like mine) have Phillips batting in the 5th spot or lower.
  • None of the lineups that bat Dunn in the #2 spot do very well, despite his crazy-high OBP.
  • My "idiot" lineups, in which I just ranked players by a rate stat like OPS, did pretty well for themselves (better than my "smart" ones). Ranks by AVG did particularly well, despite batting Dunn 8th!
Markov also can report how often each lineup slot will lead off an inning for any lineup configuration. Here is Baker's lineup, as well as the top-5 and bottom-5 lineups, broken down by how many innings each slot led off (on average):

Lineup Slot - # Innings led off
Lineup Name 1 2 3 4 5 6 7 8 9 StDev
Baker-OD 1.76 0.77 0.81 0.95 1.04 0.86 0.89 0.85 1.06 0.30
Top 5 Lineups










Bluzer-OD 1.80 0.78 0.79 1.08 0.95 0.85 1.15 0.88 0.72 0.33
Chris-OD2 1.81 0.77 0.79 1.03 1.02 0.88 1.13 0.83 0.73 0.33
Pickoff-OD 1.86 0.77 0.78 1.03 0.97 0.89 1.14 0.83 0.73 0.35
AVG-rank 1.78 0.77 0.68 1.01 1.05 0.89 1.13 0.92 0.78 0.33
brad-OD1 1.8 0.8 0.8 1.1 1 0.9 1.1 0.9 0.7 0.33
Bottom 5 Lineups










Degenerate-OD 1.80 0.77 0.79 1.38 0.93 0.86 0.90 0.83 0.73 0.36
fareast-OD 1.82 0.76 0.68 1.37 0.95 0.82 0.90 0.85 0.85 0.36
brad-OD2 1.84 0.77 0.79 1.38 0.93 0.80 0.86 0.90 0.72 0.37
joel-OD1 1.82 0.82 0.67 1.37 0.96 0.80 0.87 0.90 0.78 0.37
redsmanrick-OD 1.84 0.77 0.79 1.38 0.93 0.81 0.87 0.89 0.72 0.37

Observations:
  • Baker's lineup is very different from the others:
    • It has the lowest frequency with which the leadoff hitter would lead off innings, though only by a small amount. Still, it might be enough to help diminish the problem of Patterson leading off.
    • It has the highest frequency with which the #3 hitter would lead off innings. The #3 hole is the spot in the lineup that most frequently bats with two outs and runners on in real baseball, but Baker's lineup might reduce this effect compared to the others.
    • Baker's lineup also has the pitcher leading off innings more often than any other lineup. That can't possibly be a good thing, can it?
  • Top-5 lineups vs. Bottom-5 lineups
    • The #4 slot leads off innings an awful lot for the bottom-5 set of lineups. This is typically a power hitter spot (either Griffey or Encarnacion in these cases), so you'd normally want runners on base when they're hitting. This problem is minimized in Baker's lineup.
    • One really neat finding: the standard deviation among lineup slots in the frequency with which they led off was consistently lower for "good" lineups then for "bad" ones. This might mean that the "bad" lineups have more bottlenecks that tend to kill innings, resulting in certain lineup spots being more likely to lead off the next inning. So, perhaps good lineups distribute the best hitters around the lineup more than poor ones?
...I'd like to do a bit more with these data, breaking down the different lineups based on their rate stats, but this has taken long enough to get written as it is. So, let's move on to the Best of the Organization lineups.

Best of the Organization Lineups

The other set of lineups I requested involved folks' choice of any players from within the Reds' organization. I received 26 of these lineups. Here they are, again along with Dusty Baker's opening day lineup, as well as a few of his others from the first week.

Name 1st 2nd 3rd 4th 5th 6th 7th 8th 9th R/G R/162G vBaker
mikegrayson-Best Bruce Keppinger Griffey Dunn Encarnacion Phillips Votto Ross Pitcher 5.01 811.3 22.2
brad-Best Keppinger Votto Bruce Dunn Griffey Encarnacion Phillips Valentin Pitcher 5.00 810.3 21.2
Chris-Best1 Keppinger Dunn Griffey Bruce Votto Encarnacion Phillips Valentin Pitcher 4.99 808.8 19.7
jinaz-Best-exploit Votto Dunn Bruce Encarnacion Griffey Keppinger Phillips Valentin Pitcher 4.95 802.0 12.9
ED44-Best2 Keppinger Votto Dunn Phillips Griffey Encarnacion Patterson Valentin Pitcher 4.95 801.9 12.8
Alex-Best Keppinger Votto Dunn Phillips Griffey Encarnacion Patterson Valentin Pitcher 4.95 801.9 12.8
Alex-best3 Freel Keppinger Dunn Phillips Griffey Encarnacion Votto Valentin Pitcher 4.94 799.7 10.6
Baker-Game4 Patterson Keppinger Griffey Phillips Dunn Encarnacion Votto Valentin Pitcher 4.93 798.0 8.9
ED44-Best Keppinger Bruce Dunn Phillips Griffey Votto Encarnacion Valentin Pitcher 4.92 797.7 8.6
Chris-Best2 Bruce Keppinger Encarnacion Dunn A_Phillips Griffey Phillips Hanigan Pitcher 4.90 794.0 4.9
justincredible-Best Votto Keppinger Griffey Dunn Phillips Bruce Encarnacion Ross Pitcher 4.88 789.8 0.7
mlbfan30-B Votto Keppinger Dunn Encarnacion Griffey Phillips Bruce Valentin Pitcher 4.88 789.8 0.7
Baker-OD Patterson Keppinger Griffey Phillips Dunn Encarnacion Hatteberg Valentin Pitcher 4.87 789.1 0.0
Alex-best2 Hopper Keppinger Dunn Phillips Griffey Encarnacion Votto Valentin Pitcher 4.85 785.7 -3.4
redsmanrick-best Votto Dunn Encarnacion Griffey Keppinger Bruce Phillips Ross Pitcher 4.85 785.6 -3.5
Enquirer-vsR Votto Dunn Encarnacion Bruce Griffey Phillips Valentin Keppinger Pitcher 4.84 783.9 -5.2
jinaz-Best Votto Encarnacion Dunn Phillips Griffey Bruce Keppinger Valentin Pitcher 4.83 782.0 -7.1
joel-Best1 Votto Dunn Encarnacion Griffey Bruce Phillips Valentin Pitcher Keppinger 4.81 779.8 -9.3
brad-Best Votto Dunn Encarnacion Griffey Bruce Phillips Keppinger Valentin Pitcher 4.80 778.4 -10.7
joel-Best2 Keppinger Dunn Phillips Bruce Encarnacion Griffey Ross Pitcher Gonzalez 4.80 778.2 -10.9
joel-Best3 Keppinger Dunn Phillips Encarnacion Griffey Patterson Ross Pitcher Hatteberg 4.79 776.5 -12.6
DannyB-B Keppinger Dunn Griffey Encarnacion Votto Phillips Hopper Valentin Pitcher 4.77 773.2 -15.9
texasdave-B Keppinger Votto Dunn Encarnacion Griffey Phillips Ross Hopper Pitcher 4.76 771.3 -17.8
fareast-Best Votto Phillips Griffey Dunn Bruce Encarnacion Gonzalez Ross Pitcher 4.76 770.5 -18.6
PastAndPending-B2 Freel Keppinger Bruce Griffey Phillips Dunn Votto Ross Pitcher 4.74 767.9 -21.2
Bluzer-B Hopper Keppinger Dunn Phillips Griffey Encarnacion Valentin Pitcher Freel 4.74 767.4 -21.7
Baker-Game2 Patterson Keppinger Griffey Phillips Dunn Encarnacion Votto Bako Pitcher 4.73 766.3 -22.8
PastAndPending-B Hopper Keppinger Griffey Phillips Dunn Votto Encarnacion Valentin Pitcher 4.72 764.9 -24.2
JamesB-B Hopper Keppinger Griffey Dunn Phillips Votto Encarnacion Valentin Pitcher 4.72 764.4 -24.7
redsmanrick-best2 Keppinger Phillips Dunn Encarnacion Griffey Ross Votto Pitcher Hopper 4.71 763.2 -25.9
DannyB-B2 Hopper Dunn Griffey Encarnacion Votto Phillips Gonzalez Ross Pitcher 4.69 759.3 -29.8
Enquirer-vsL Keppinger Dunn Phillips Bruce Encarnacion Griffey Ross Gonzalez Pitcher 4.68 758.7 -30.4
Baker-Game3 Freel Keppinger Griffey Phillips Dunn Encarnacion Hatteberg Bako Pitcher 4.63 750.1 -39.0

Thoughts:
  • Player choice matters: here we see that Baker's opening day lineup can be beat regularly by employing Joey Votto and Jay Bruce in favor of Scott Hatteberg and Corey Patterson.
  • The difference between the best and worst lineups in this case was about 5 wins. That's lower than I expected given the 4-win range in the opening day dataset, but then again the personnel differences among these lineups aren't that dramatic.
  • I was gratified to see that my "exploitative" lineup, which takes advantage of the lack of information in the model about L/R splits and strings together Votto, Dunn, and Bruce in the top-3 slots, did quite well. So I'm not completely hopeless...
  • Mike Grayson's top-rated lineup has Jay Bruce and his projected 0.336 OBP in the leadoff slot, again indicating that the issue of OBP in the 1-hole is less of a big deal than it's often made out to be.
Let's look at the distribution of how often hitters lead off innings from this dataset and see if we see a similar trend to the prior dataset:

Lineup Slot - Innings led off
Lineup Name 1 2 3 4 5 6 7 8 9 StDev
Baker-OD 1.76 0.77 0.81 0.95 1.04 0.86 0.89 0.85 1.06 0.30
Top 5 lineups









mikegrayson-Best 1.77 0.77 0.78 0.94 0.98 0.88 0.91 0.90 1.06 0.30
brad-Best 1.80 0.78 0.77 1.07 0.89 0.88 1.17 0.88 0.77 0.33
Chris-Best1 1.79 0.77 0.77 1.03 0.89 0.88 1.17 0.87 0.83 0.32
jinaz-Best-exploit 1.78 0.77 0.77 1.38 0.87 0.89 0.87 0.88 0.79 0.35
ED44-Best2 1.80 0.78 0.79 1.08 1.02 0.83 1.14 0.84 0.72 0.33
Bottom 5 lineups









JamesB-B 1.77 0.78 0.79 1.03 0.97 0.79 0.91 0.92 1.05 0.31
redsmanrick-best2 1.79 0.80 0.73 0.95 1.06 0.91 1.12 0.84 0.80 0.32
DannyB-B2 1.75 0.76 0.78 1.03 1.02 0.88 0.77 0.93 1.07 0.30
Enquirer-vsL 1.83 0.76 0.67 1.36 0.95 0.81 0.89 0.91 0.82 0.37
Baker-Game3 1.76 0.77 0.78 0.95 1.05 0.86 0.91 0.87 1.07 0.30

Not the same trend. Mostly. The top lineups continue to have a fairly low standard deviation (at least compared to the bottom 5 from the opening day lineups), but we see low variation here among the bottom lineups as well. What's different?

Well, the bottom lineups tend to have inferior players to the better lineups this time around. Norris Hopper appears frequently instead of Corey Patterson in these lineups, which is a poor trade according to PECOTA. And Baker's game 3 lineup includes Freel and Bako instead of Patterson and Valentin.

...

Well, there's obviously a lot more to do on this front. I feel like we've really just started to scratch the surface on this issue. But I wanted to end with a summary of some of the tentative conclusions that I'm taking from this work.

1. There isn't one best way to make a successful lineup. Lineup interactions are such that there may be several very different styles of lineups that each result in similar overall performance.

2. In general, playing the best players is more important than lineup order. That's not to say that lineup order doesn't matter, but a poorly structured lineup with great players can beat a perfectly structured lineup with weak players.

3. Spreading one's best (and worst) hitters out to prevent bottlenecks may be an overlooked yet highly influential means of improving the performance of a lineup. The first dataset indicates to me that it might be more important than some of the other things we tend to worry about (like OBP in the leadoff slot). At the very least, it's worth further study.

4. Seemingly small differences between players can cause substantial differences in lineup performance when they are swapped between lineup spots. It may be that different sets of players may require substantially different lineup designs for optimal production.

5. We fans may not know as much about lineup construction as we think we do. In my case, at least, I'm feeling pretty clueless after working through this project. This post by J.C. Bradbury seems pretty apt. So, I'm inclined to give Dusty Baker the benefit of the doubt with his lineup order at this point...though I reserve the right to complain about personnel choices!


That's about as far as I'm willing and able to go for now. If you have suggestions, specific tests (or specific lineups) you'd like to see done, etc, let me know. I'm happy to continue working on this stuff as my (limited) time permits.

A few caveats and notes:
  • All of the above results are obviously dependent on the quality and reliability of the Markov model upon which they're based. It seems really solid, but we should always keep this in mind. Again, I'll refer to the Bradbury post.
  • As implied above, a different set of players--say, #1-#9 lineup splits for NL teams...not to mention AL teams--might result in very different findings. Such is the nature of this sort of thing.
  • For pitchers and their pinch hitters, I just used 2007 Reds' pitcher totals. Maybe I should have used #9 hitter totals instead, as they would recognize the contributions of pinch hitters. But I didn't think about that until about a minute before hitting "publish post."
  • With some assistance from John Beamer (thanks John!), I modified the spreadsheet to automatically input the actual % times each hitter led off an inning, based on the output of the model. You'll need to do this as well if you want to replicate my results. Drop me a line and I can help ya out on that.
  • I used the standard baserunning tables in the spreadsheet, and I did click the Update SBA button after constructing each lineup. I have no idea how much modifying those baserunning tables might affect the results.
  • I also just wanted to again thank John Beamer for publishing his Markov model with the Hardball Times Annual. If nothing else, it's provided a lot of food for thought!
Photo by Getty Images/Jonathan Daniel

Additional discussion about this project can be found at: