Catchers play the most unique of all eight non-pitcher positions. Many of the skills required to be an effective catcher--calling a game, handling a pitcher, etc--are arguably unique from those required to be effective at other positions. Furthermore, their performance is tightly intertwined with the performance of their pitchers, which makes evaluating catchers harder than other positions, often requiring multiple seasons of data to do well.Intuitively, one approach you might take is to try to assess a catcher's performance by assessing how pitchers do when interacting with that catcher. This is the reasoning behind catcher ERA (cERA), which tracks the ERA of pitchers while a particular catcher is behind the plate. Nevertheless, in Baseball Between the Numbers, Keith Woolner gave an overview of some of the past research evaluating catchers' influences on their pitchers' performance. Surprisingly, he reported that there was little if any consistent skill that catchers have on their pitchers. For example, cERA did not vary from overall team ERA in a manner that showed any predictive direction from year to year.
Catchers do apparently vary consistently, however, in statistics that are more directly under their control: throwing out baserunners, preventing wild pitches and passed balls, and avoiding errors. To be sure, the pitchers that a particular catcher is receiving can have a substantial effect on these rates. For example, Doug Mirabelli is likely to have a high WP+PB rate simply because he catches knuckleballer Tim Wakefield so often. Furthermore, left-handers are well-documented to hold runners better than right-handers. Nevertheless, catchers do seem to vary in consistent ways with these skills, so I think it's worthwhile to track them. What I describe below is a quick and unsophisticated way of doing this. Hopefully, in the future, I can expand upon this and make it better--but for now, I'd guess that it works pretty well.
For all of this work, I'm using The Hardball Times' catching statistics, which are the best easily-accessible data source on catching that I'm aware of. Here's how I'm estimating runs saved for each variable:
Runs saved via stolen bases
First, I calculate average caught stealing rate across MLB catchers as:
lgCSRate = CS/SBA
where SBA is stolen base attempts. You'll note that we have to back-calculate SBA, SB, and CS values from the reported SBA/9innings, CS%, and Inning values reported by THT. In 2007, lgCSRate = 0.22 CS/SBA.I then find each catcher's +-caught steals by:
+-CS = CS - (lgCSRate*SBA)
where CS and SBA are the player's values.Finally, I convert these value to a +-Runs value by:
+-CSRuns = +-CS * 0.63
where 0.63 is the difference in runs (according to linear weights) between a stolen base (0.19 runs scored) and a caught stealing (-0.44 runs scored). This is the "swing" in runs scored between allowing a stolen base and gunning down a runner.Runs saved via wild pitches and passed balls
I do this via the same general approach as above. First, I calculate league average wild pitch plus passed ball rate as:
lgWPPBRate = (WP+PB)/Inn
where WP+PB is the total number of wild pitches and passed balls in MLB during the season in question, and Inn is the total number of innings caught by all catchers. We have to back-calculate WP+PB from the WP+PB/9inning values reported by THT. Also, ideally, I'd use pitches caught instead of innings in the denominator, but I don't have those data on hand. So anyway, in 2007, lgWPPBRate = 0.042 WP+PB/Inn.Next, I find each catcher's +-[wild pitches plus passed balls]:
+-WPPB = (WP+PB) - (lgWPPBRate*Inn)
where WP+PB and Inn are the player's values.Finally, I convert this value to a +-Runs value by:
+-WPPBRuns = +-(WP+PB) * 0.28 * -1
where 0.28 is the average value of a wild pitch or passed ball, according to linear weights. The alternative to a wild pitch or passed ball is no change in base runner or out status (usually), so the straight-up linear weights value of a PB or WP is all we need. I multiply by -1 to make this a runs saved value rather than a runs allowed value.Runs saved via errors
The final component to "my" catcher fielding estimates is runs saved via errors.
First, I calculate average throwing error (TE) and fielding error (FE) rates across MLB catchers as:
lgTERate = TE/Inn
and
lgFERate = FE/Inn
where TE, FE, and Inn are MLB totals. In 2007, lgTERate = 0.0053 TE/Inn, while lgFERate = 0.0015 FE/Inn.and
lgFERate = FE/Inn
I then calculate a player's +-TE and +-FE values as:
+-TE = TE - (lgTERate*Inn)
and
and
+-FE = FE - (lgFERate*Inn)
where TE, FE, and Inn are the player's values.
Finally, I convert these value to a +-Runs saved value like this:
+-TERuns = +-TE * 0.48 * -1
and
+-FERuns = +-FE * 0.75 * -1
Using the procedure outlined above, below are fielding estimates for 2007 catchers with a minimum of 400 innings behind the plate. FWIW, the correlation between the FSR data and the sum of the empirical fielding ratings (listed below as +-Runs) was 0.60, which is encouraging.and
+-FERuns = +-FE * 0.75 * -1
You'll note that I'm using different runs values for TE's and FE's. Here's my reasoning. Fielding errors are usually made on plays that would otherwise be outs. For example, a catcher not being able to handle a good throw to the plate results in an error that would otherwise have resulted in an out. Therefore, the run value of making one error above average is the value of the advancing runners (0.48 runs allowed) plus the value of the out (0.27 runs saved), or 0.75 runs total cost to the team.
On the other hand, many catcher throwing errors are made on stolen base attempts, with the runner usually ending up at third. These plays are typically scored as a stolen base plus an error. Therefore, the difference between making the error and not making the error is just the advancement of the runner(s) (~0.48 runs allowed); we can't assume than an out would have been made. It's true that some throwing errors would have resulted in outs (e.g. throwing away a ball on an easily-fielded bunt), but I tend to err on the side of being conservative with fielding statistics. I am, however, very much open to suggestions on how to better handle this issue.
Note: After I completed the above work, I discovered (thanks to a tip by MB) this article by Chone Smith. He describes a very similar methodology for evaluating catchers, though mine differs in two small ways. First, I compare each player's CS's to league average caught stealing rate rather than just using raw runs values for SB's and CS's. I do this because I'm interested in fielding relative to the competition. Second, I don't adjust for the rate at which steals are attempted because I think there are probably too many other factors that can influence that rate. Chone also uses a 0.48 run value for errors, which is encouraging...though I still think an 0.75 run value for fielding errors is appropriate.
Fans' Scouting Report
One additional resource that we have available to us in evaluating catchers is the Fans' scouting report, which, as with players at other positions, can be converted to an approximate +-runs statistic using the skill weightings provided by Tom Tango as well as the assumption that each point is worth ~0.7 runs. These +-runs ratings should be pro-rated relative to the number of innings a player caught out of 1440 (~162 games worth of defensive innings).
These scouting measures are very useful, but my tendency is to down-weight them in recognition of their basis in the subjective impressions of (usually) untrained fans. Therefore, consistent with how I used FSR data with other position players, I'm estimating overall catcher fielding as:
2007 Catchers
On the other hand, many catcher throwing errors are made on stolen base attempts, with the runner usually ending up at third. These plays are typically scored as a stolen base plus an error. Therefore, the difference between making the error and not making the error is just the advancement of the runner(s) (~0.48 runs allowed); we can't assume than an out would have been made. It's true that some throwing errors would have resulted in outs (e.g. throwing away a ball on an easily-fielded bunt), but I tend to err on the side of being conservative with fielding statistics. I am, however, very much open to suggestions on how to better handle this issue.
Note: After I completed the above work, I discovered (thanks to a tip by MB) this article by Chone Smith. He describes a very similar methodology for evaluating catchers, though mine differs in two small ways. First, I compare each player's CS's to league average caught stealing rate rather than just using raw runs values for SB's and CS's. I do this because I'm interested in fielding relative to the competition. Second, I don't adjust for the rate at which steals are attempted because I think there are probably too many other factors that can influence that rate. Chone also uses a 0.48 run value for errors, which is encouraging...though I still think an 0.75 run value for fielding errors is appropriate.
Fans' Scouting Report
One additional resource that we have available to us in evaluating catchers is the Fans' scouting report, which, as with players at other positions, can be converted to an approximate +-runs statistic using the skill weightings provided by Tom Tango as well as the assumption that each point is worth ~0.7 runs. These +-runs ratings should be pro-rated relative to the number of innings a player caught out of 1440 (~162 games worth of defensive innings).
These scouting measures are very useful, but my tendency is to down-weight them in recognition of their basis in the subjective impressions of (usually) untrained fans. Therefore, consistent with how I used FSR data with other position players, I'm estimating overall catcher fielding as:
+-Fielding = .75*([+-CSRuns] + [+-WPBPRuns] + [+-TERuns] + [+-FERuns]) + .25*FSR
2007 Catchers
My estimates put the difference between the best (Yadier Molina) and worst (Josh Bard) catchers in 2007 as being ~20 runs, or about two wins. This is less of a difference than we see among other positions, but remember that catchers usually don't play as many innings as players at other positions. I'm also probably not accounting for all the ways that catchers may differ. But I think we're capturing at least an important component of catcher defensive abilities.
| Last | First | Tm | Inn | CSRns | WPPBRns | TERns | FERns | +-Runs | FSR | +-Fielding |
| Molina | Yadier | STL | 861 | 8.1 | 1.6 | -0.2 | 0.2 | 9.7 | 17.6 | 11.5 |
| Johjima | Kenji | SEA | 1107 | 8.2 | 0.7 | 2.8 | 0.5 | 12.1 | 5.1 | 10.3 |
| Mauer | Joe | MIN | 778 | 6.5 | 0.0 | 1.5 | 0.8 | 8.9 | 15.2 | 10.3 |
| Laird | Gerald | TEX | 987 | 11.1 | 0.0 | -1.8 | -1.2 | 8.1 | 9.6 | 8.4 |
| Varitek | Jason | BOS | 1064 | 0.5 | 6.9 | 0.8 | -0.3 | 7.9 | 6.3 | 7.4 |
| Redmond | Mike | MIN | 483 | 3.2 | 3.5 | 1.2 | 0.5 | 8.4 | 4.4 | 7.4 |
| Martin | Russell | LAN | 1254 | 4.3 | 2.3 | -3.5 | 1.4 | 4.5 | 16.5 | 7.3 |
| Ross | David | CIN | 837 | 6.3 | -0.8 | 0.7 | 0.2 | 6.3 | 6.9 | 6.4 |
| Snyder | Chris R | ARI | 891 | 1.8 | 0.3 | 1.8 | 1.0 | 4.8 | 11.7 | 6.4 |
| Schneider | Brian | WAS | 1051 | 2.8 | 0.3 | -0.2 | 1.1 | 4.0 | 12.7 | 6.1 |
| Martinez | Victor | CLE | 1043 | 5.0 | 3.2 | 2.2 | -1.1 | 9.3 | -4.2 | 6.0 |
| Ruiz | Carlos | PHI | 913 | 1.4 | 1.7 | 1.4 | 1.0 | 5.5 | 6.8 | 5.8 |
| Ausmus | Brad | HOU | 907 | -2.4 | 5.9 | 0.4 | 1.0 | 4.9 | 7.9 | 5.5 |
| Miller | Damian | MIL | 446 | 2.0 | -0.3 | 1.1 | 0.5 | 3.3 | 2.9 | 3.2 |
| Shoppach | Kelly B | CLE | 420 | 3.2 | 0.3 | -0.4 | -0.3 | 2.8 | 3.9 | 3.0 |
| Rodriguez | Ivan | DET | 1053 | 2.9 | -4.3 | 0.8 | -0.4 | -1.0 | 13.8 | 2.6 |
| Iannetta | Chris D | COL | 497 | -0.5 | 2.2 | 0.8 | 0.5 | 3.0 | 0.1 | 2.3 |
| Torrealba | Yorvit | COL | 935 | -1.9 | 3.5 | -1.0 | 1.0 | 1.7 | 2.1 | 1.8 |
| Lo Duca | Paul | NYN | 974 | -1.7 | 4.2 | -1.8 | 1.1 | 1.8 | -2.5 | 0.7 |
| Suzuki | Kurt K | OAK | 539 | -0.7 | -1.5 | 0.9 | 0.6 | -0.7 | 2.5 | 0.1 |
| Hernandez | Ramon | BAL | 855 | -1.1 | 0.0 | -0.2 | -0.6 | -1.9 | 4.9 | -0.2 |
| LaRue | Jason | KC | 474 | 2.7 | -1.2 | -0.2 | -1.0 | 0.3 | -2.6 | -0.4 |
| Paulino | Ronny | PIT | 1088 | -1.2 | 2.0 | 1.3 | -1.1 | 1.1 | -6.0 | -0.6 |
| Buck | John R | KC | 924 | -1.7 | 1.4 | 0.0 | -0.5 | -0.8 | -0.2 | -0.6 |
| Molina | Bengie | SF | 1104 | 0.9 | -2.8 | -0.5 | 0.5 | -2.0 | 2.5 | -0.9 |
| Treanor | Matt A | FLA | 441 | -2.7 | -0.1 | -0.3 | 0.5 | -2.7 | 1.5 | -1.7 |
| Montero | Miguel | ARI | 511 | -0.6 | -0.5 | -0.1 | -0.9 | -2.1 | -0.5 | -1.7 |
| Pierzynski | A.J. | CHA | 1058 | -2.8 | -0.3 | 1.7 | 1.2 | -0.2 | -6.4 | -1.7 |
| McCann | Brian | ATL | 1139 | -1.1 | 1.8 | -0.9 | -2.5 | -2.8 | 1.4 | -1.8 |
| Kendall | Jason | OAK | 714 | -2.6 | 0.2 | 0.4 | 0.0 | -2.0 | -2.6 | -2.1 |
| Napoli | Mike A | LAA | 599 | -0.4 | -1.1 | 0.6 | -2.3 | -3.3 | 0.9 | -2.3 |
| Navarro | Dioner | TB | 956 | 1.8 | 0.3 | -3.3 | -0.5 | -1.7 | -4.8 | -2.4 |
| Posada | Jorge | NYA | 1111 | 0.0 | -5.2 | 1.9 | -1.0 | -4.4 | 3.3 | -2.5 |
| Mathis | Jeff | LAA | 467 | -1.5 | -2.6 | -0.2 | -0.2 | -4.6 | 3.8 | -2.6 |
| Zaun | Gregg | TOR | 838 | -4.3 | 2.9 | -1.2 | 0.2 | -2.5 | -5.5 | -3.2 |
| Valentin | Javier | CIN | 472 | -3.1 | -1.2 | 0.7 | 0.5 | -3.1 | -4.4 | -3.4 |
| Bako | Paul | BAL | 421 | -0.9 | -1.7 | -0.8 | 0.5 | -3.0 | -4.9 | -3.4 |
| Barrett | Michael | CHN | 475 | -3.4 | -0.9 | -0.2 | -0.2 | -4.8 | -6.4 | -5.1 |
| Kendall | Jason | CHN | 432 | -6.2 | -0.8 | -1.3 | 0.5 | -7.8 | -1.6 | -6.2 |
| Olivo | Miguel | FLA | 990 | 2.7 | -7.1 | -1.3 | -1.9 | -7.6 | -2.3 | -6.3 |
| Estrada | Johnny | MIL | 961 | -7.0 | 0.0 | 0.5 | 1.0 | -5.4 | -16.0 | -7.9 |
| Bard | Josh | SD | 927 | -13.0 | 2.6 | 1.4 | 0.3 | -8.7 | -5.7 | -7.9 |
Photo of Yadier Molina by AP/Rick Bowmer
I was curious about one thing. In an earlier segment of this series you ranked the various positions in the context of fielding. For example you indicated that CF was a +5, SS a +4 and so on. You didn't give a +/- for the catching position. Is there one? Maybe I missed it somewhere along the line. And, if there is one, what is it? Thanks.
ReplyDeleteYeah, that's a problem. Most analysts seem to agree that catching is the hardest job on the field, and that there's a very small pool of players that can fill that role adequately.
ReplyDeleteHowever, because it's such a unique job, and tends to have such unique players, catchers a) don't tend to move positions much, and b) don't tend to do particularly well when moved to another position (Craig Biggios and Brandon Inges aside). So you'll see C/1B's and C/3B's, but rarely are they particularly good when moved to those positions, which means that a position ranking based on changes in performance will rank catchers as a rather "easy" position defensively.
Given how terrible their hitting is, we either have to accept that catchers are a massively inferior position in terms of talent, or just try to ballpark our position adjustment. In Tom Tango's post on defensive spectrum, he put catchers at +10 runs/season "purely by intuition," which is twice the adjustment of the next toughest position (CF). That's what I'm using for the time being, as shaky as it is.
One thing we could perhaps try to do is look at offensive disparities, because presumably everyone who can play catcher is already playing catcher. And catchers, on average, hit 4.1 r/g, which is ~1 r/g below league average. Over a full season, that translates to ~15 runs per season less. So we could use that number as our adjustment, but it's three times the magnitude of the next largest adjustment...and I'm just not willing to go there yet.
-j
Awesome, obviously.
ReplyDeleteAs for the catcher adjustment, has anyone compared hitting by catchers to seasons when they didn't play catcher? Most of those would come as the catcher got older, but you could apply a generic aging curve. The sample size would be decent, given all the catchers who have become 3B/1B/DHs. Offensive positional adjustments aren't the best route, as we've discussed, but it's probably worth taking a look at for catchers.
Justin, any thoughts on adjusting the caught stealing numbers based on some sort of staff adjustment. I can't think of how you'd come to that number, but it seems to heavily penalize a catcher who catches a guy like Maddux who is slow to the plate and doesn't do much to hold runners.
ReplyDeleteCertainly the pitcher(s) have some real affect on CS%.
Hi Rick,
ReplyDeleteI actually think a bigger problem (and more easily corrected problem) is the influence of the handedness of pitchers on a staff, as lefties as a whole hold runners much better than righties. David Gassko has apparently worked an adjustment like this into his work on catchers, but I haven't figured out an efficient way to do this.
Another big issue has to do with the type of pitches that a catcher is receiving, especially with knuckleballers, but also probably with junkballers vs. hard throwers. The pitchf/x data might help with that eventually.
Working around delivery time to the plate would seem to me to be among the hardest issues to deal with. That probably is best solved by comparing catchers within pitchers--how well does Josh Bard, for example, do on CS% with Maddux compared to other Maddux catchers. But that makes season estimates of performance pretty hard to do...
-j
Funny thing... I was going to say something about the pitchf/x data earlier. It'd be interesting to see the average speed per pitch on each steal attempt. For instance, against one catcher runners may guess right often and go on many curves or off speed stuff, while against another they may happen to go on mostly fastballs. You could probably make an adjustment like that, or do it in a more detailed approach (perhaps based on pitch speed, location,pitch type, as well as handedness as you suggest).
ReplyDeleteAnyway, great work as usual. I'm glad my comment in the other thread led you to that article, although it looks like you had things mostly finished anyway. I'm not sure about the difference in allowing stolen bases against, like what Chone did. I would think it's a real skill, yet I'm sure there are a lot of variables at play like you suggest (like there are with many of these things).
FWIW, on my other blog, I calculated that (just using the number in Chone's article) for Yadier Molina and Russ Martin and the difference was about 3 runs (going in Molina's favor).
Great Stuff. Too bad this doesn't get out before the Gold Gloves are issued.
ReplyDeleteHeh, Ivan Rodriguez will keep winning that award for as long as he continues to play. :) -j
ReplyDelete