The Commissioner Speaks: Imagining a Redefined Strike Zone

Commissioner Rob Manfred has been keeping an eye on the strike zone this year. (via Arturo Pardavila III)

Before the season, Yahoo’s Jeff Passan reported that the strike zone would be monitored this year by the MLB Playing Rules Committee. With run scoring falling to its lowest level since 1981, the committee was thought to have the potential to recommend a change to the rulebook strike zone as early as 2016 if it deemed such a move prudent for the welfare of the game.

Eno Sarris recently had the opportunity to ask new Commissioner Rob Manfred about the strike zone, and this is what he had to say:

It’s a work in progress (referring to fact that the zone got smaller and then bigger over the course of the first two months). First of all, all of these changes you’re talking about are minuscule in terms of magnitude. Having said that, there has been absolutely no direction given to the umpires this year that is different from what we’ve given them the last few years, which is to call the strike zone consistent to the rules. Our focus with respect to the strike zone has not been on changing it in any way; to the contrary, it has been on making it more consistent across umpires. We think we’ve made progress in that regard.

Conceptually — that was the facts, what’s out there — conceptually, I would be reluctant, even if someone said you have X problem, and the way to fix it is to give instructions on changing the strike zone, I would be reluctant to do that. Because when you talk to baseball people, baseball people you respect, and you say to them what happens if I do this with the strike zone, you don’t get any consistent response. I am very disciplined to make changes with respect to the play of the game on the field where we don’t have a pretty good idea of what the outcome is going to be.”

There is a lot of information in this quote from the commissioner, so let’s examine his statements piece by piece.

Magnitude of Changes

It’s a work in progress (referring to fact that the zone got smaller and then bigger over the course of the first two months). First of all, all of these changes you’re talking about are minuscule in terms of magnitude.”

When I first measured the 2015 season strike zone near the end of April, both its overall size and the band at the bottom of the zone where expansion clearly has been visible had both contracted ever so slightly. By the time the end of May had rolled around, the strike zone had filled out to narrowly exceed the zone measured last season.

Measuring now at the midway point in the season shows results similar to last check.

I would agree that the magnitude of the changes seen in these measurements is small when placed up against the 2014 numbers. The overall size is identical, with the size of the zone below 21 inches only 2 square inches larger. What I would claim to be the larger potential issue is that the size and shape of the strike zone in today’s game is drastically different from where it was five or six years ago. I’ll return to this subject a little later. Let’s continue parsing commissioner Manfred’s quote.

Direction Given to Umpires

Having said that, there has been absolutely no direction given to the umpires this year that is different from what we’ve given them the last few years, which is to call the strike zone consistent to the rules.”

This statement matches the sentiment reported by Jeff Passan (linked above), namely that the MLB Playing Rules Committee “will pay close attention to the size of the strike zone in 2015 with an eye on change as early as 2016.” There is no reason to think umpires have been asked to call the strike zone any differently in 2015 than they were in 2014, and this season’s zone as I measure it looks very similar to last season’s zone.

Consistency Across Umpires

Our focus with respect to the strike zone has not been on changing it in any way, to the contrary, it has been on making it more consistent across umpires. We think we’ve made progress in that regard.”

Fellow strike zone researcher Brian Mills has done excellent work in the area of home plate umpire ball/strike calling accuracy and consistency. In the introduction of his peer-reviewed journal article, “Expert Workers, Performance Standards, and On-the-Job Training: Evaluating Major League Baseball Umpires,” he writes this about the state of umpiring accuracy during the PITCHf/x era.

Additionally, umpires have shown substantial improvement in their ball-strike call accuracy. In 2007, umpires correctly called only 76.8% of pitches within the rulebook zone and 87.4% of pitches outside the rulebook zone based on the Sportvision data. However, by 2013, umpires were correctly calling 85.7% and 90.0% of pitches within and outside the zone, respectively.”

Using the zone definitions provided in the Gameday data, I compared umpire pitch calling accuracy over our period of study.

The results agree with Mills’ assessment that pitch calling accuracy is trending in the right direction.

If you’re surprised at how high the percentages are for incorrect calls, I believe the main culprit at play is the way that the zone is called differently depending on game state. While I believe Gameday uses constant zone definitions for all pitches in a given season, many researchers (including myself) have shown that the zone expands and contracts based on game state, in particular by count. As a quick example, if I look at only pitches on 0-0 counts from 2014, the percentage of balls called strikes jumps to 20.2 percent, but the percentage of strikes called balls drops to 7.5 percent. This is consistent with the previous research which shows a 0-0 count is pitcher-friendly as far as the size of the strike zone is concerned.

In Table 2 of his paper, Mills calculates a Weighted Coefficient of Variation (wCV), his measure of strike variability between umpires by season. A lower wCV is indicative of lower variation among umpires. As his paper was written in 2014, he calculated this measure for seasons up to and including 2013. After reproducing his 2013 value, I calculated the same measure for 2014.

Where Mills left off, there had been a clear trend toward better overall consistency between umpires across the league that actually extended back to 1999, when wCV was much higher at 0.01554. The measure of variability from 2014, while not another improvement, appears to be in line with recent seasons.

Out of interest, I calculated the percentage of pitches thrown when a “new” umpire was behind the plate, where “new” here is defined as umpires who did not call any major league games at all in the previous season. The results shows variability across the league held quite steady despite a relatively large influx of new umpires last season. I would suggest this speaks well of the Zone Evaluation system implemented to assess umpires across the league.

To compare umpire consistency fairly for 2015, I measured the same values for previous seasons, but only for all games up until the end of June.

Notice the values are higher than for full seasons, when the pools of pitchers that each umpire calls from behind the plate have time to distribute more evenly. While variability this season is certainly no better than recent years, it does not look like there is any significant degradation here.

For one additional viewpoint, I calculated the same measure across the seasons of study using only called strikes as a percentage of called pitches (wCVCP), as opposed to counting all non-balls as strikes in the previous metric (wCV).

The scale changes when looking only at called pitches, but again it would appear the last several years have produced less variable results than earlier on in the Zone Evaluation era.

One observation I’ve made in my research with respect to umpire consistency is that there is an example of umpire cohorts adjusting to a change in the strike zone at different rates. In looking at the “lefty strike,” the outside edge of the strike zone off the plate for left-handed hitters, I noticed a quicker pace of contraction made by newer umpires compared to their colleagues who debuted in the league prior to the 2001 strike zone re-focusing effort. For this table I have included the 2007 and 2008 seasons as well, as they provide some context to the more equal starting point for the two cohorts prior to the introduction of Zone Evaluation in 2009, as well as the magnitude of the contraction in this area during the PITCHf/x era.

There is a definite convergence happening here for all umpires toward eliminating the “lefty strike.” What can be seen, however, is that senior umpires have been contracting this region more slowly than their junior brethren. The difference is not staggering, but it is certainly present and quite consistent in magnitude since 2009.

The difference between the two umpire groups on the right-handed hitter outside edge has been much more muted, and there has been virtually no difference between the rates of change in the rapidly expanding bottom of the zone. This observation about the outside region of the left-handed hitter strike zone is my sole example of a place where umpire-to-umpire consistency may have room to improve.

So on the whole, the numbers back the commissioner’s claim that there’s been progress in regard to more consistency between umpires.

Effect of Changing the Strike Zone Definition

Conceptually – that was the facts, what’s out there – conceptually, I would be reluctant, even if someone said you have X problem, and the way to fix it is to give instructions on changing the strike zone, I would be reluctant to do that. Because when you talk to baseball people, baseball people you respect, and you say to them what happens if I do this with the strike zone, you don’t get any consistent response. I am very disciplined to make changes with respect to the play of the game on the field where we don’t have a pretty good idea of what the outcome is going to be.

Finally we come to the most open-ended aspect of the quote, thinking about what effects may be seen in the game should the strike zone definition be altered. Manfred says that people around the game have given him a variety of responses when prompted to imagine baseball with a new zone. Let’s try to talk through some potential outcomes, in as quantitative a manner as possible.

For the purposes of this exercise, the change to the strike zone that will be considered is elevating the bottom of the zone from its current level of around 18 inches off the ground (roughly the hollow of the knee) back up to its former level (in the 2009 timeframe) of 21 inches above home plate (more or less the top of the knee). The remainder of the strike zone will remain as it is being called today.

Umpire Accuracy and Consistency

If the official strike zone definition were to change, it would require ratification by the World Umpires Association. So we can realistically assume most members would be okay with the idea of a new definition. The question is, what could we expect as far as accuracy and consistency among major league umpires?

I submit that recent history tells us the Zone Evaluation system employed by MLB to assess home plate umpires works quite well, even in the face of change. Earlier we saw that as the zone has been changing and expanding at the bottom, both umpire accuracy and umpire-to-umpire consistency have been improving in recent years.

The past two and a half seasons also have witnessed much higher rates of new umpires calling games than in the previous stretch of the same length, yet umpire performance has not notably been affected by the turnover. This suggests the standard evaluation system is effective enough to ensure conformance to the desired called strike zone.

The observation about umpire cohorts adjusting to the left-handed hitter outside edge region at different rates has not translated to rates of change at the bottom of the zone. So the proposed zone change for this exercise should not introduce further umpire consistency issues along this avenue.

These facts make me believe that if the rulebook definition were to be altered, major league umpires as a group would adjust quite well, with the league’s Zone Evaluation and feedback process the key to success.

Strikeout and Walk Rates

When I first analyzed recent changes to the strike zone, I identified three regions where the majority of the changes were taking place: the outside corner for both left-handed hitters and right-handed hitters, and the bottom of the strike zone.

The outside corners for both left-handed and right-handed batters have been contracted in recent years. This has been especially prudent for left-handed hitters, who have had to deal with a strike zone shifted two to three inches off the outside edge of the plate.

The rule change proposed here only raises the bottom of the zone that has fallen so dramatically; the improvements to the zone made on the outside edges will be assumed to remain constant.

Consider first the direct changes in called strikeout and walk rates due to the way the bottom of the strike zone has been altered in recent years.

In addition to this direct impact, changes to the zone drive adaptations to batter swing rates, so there are secondary effects such as swinging strikeout rates.

Through previous research, I have shown that both pitchers and hitters learn where the zone is contracting and expanding relatively quickly. This is visible in the swinging strikeout totals on pitches in the expanded bottom of the strike zone.

I would expect the number of strikeouts and walks on pitches to the bottom of the zone to regress back toward their totals for the 2009 season, given that the strike zone definition that year had its bottom established at the same place the rulebook would define it in this exercise.

Summing up the tables gives an estimate of the change in strikeouts and walks that may arise from this zone adjustment. If we apply these changes to the 2014 totals, it would reduce the strikeout rate from 20.4 percent to 19.7 percent and increase the walk rate from 7.6 percent to 7.9 percent. The resulting (K-BB)% of 11.8 percent matches the 2012 rate, when about 4.3 runs per team per game were being scored in the league as opposed to about 4.1 runs today.

Run Scoring

If the rulebook strike zone definition is amended, it is with increased run scoring in mind as an effect. In my previous strike zone research, I estimated changes to the run environment caused by the morphing zone by calculating expected run differences for each pitch in a given season.

The idea behind this is that starting in a given count, if the next pitch is called a ball, the count moves more into the hitter’s favor, and the expected number of runs that will score from that point forward increases by a small fraction of a run. Conversely, if the next pitch is a strike, the pitcher improves his position, and the expected number of runs scored slides slightly lower.

To determine the expected number of runs for each count, I calculated the weighted on-base percentage (wOBA) for all plate appearances in a season in which a particular count was reached at any point, then divided the change in wOBA when the next pitch was a ball or strike by the yearly wOBA constant to arrive at an expected run difference in each of these cases.

With the run values calculated, I can sum up the small values over all pitches to our 18-21 inch band of interest for this exercise in both 2009 and 2014.

If the bottom of the strike zone reverted to its 2009 height, this analysis estimates about 1,000 additional runs would be scored over the course of the season. If we augment the 2014 run total by this difference, it brings the runs scored per team per game up from 4.07 to 4.27. Once again, this closely matches the run environment experienced in the league in 2012.

To this point I have considered effects that such a strike zone re-definition may have on the major leagues as a whole. In addition, we can imagine impact felt at the individual and team levels.

Pitchers Who Target the Expanded Region of the Strike Zone

Intuitively, it seems that the pitchers who would have the most to lose if the bottom of the strike zone were lifted up again would be those who pitch most often to the area being contracted. If pitches in a particular region are suddenly being called balls, my earlier research has shown, hitters learn to swing less often at such pitches in response. Concurrently, pitchers adapt and throw less frequently to the suddenly unfriendly area.

For this portion of the analysis, I’m not so concerned with who was throwing to this 18-21 inch band several years ago; what is relevant is to consider who is doing so now to get an understanding of who would be forced to adapt the most to the rule change.

Here are the 2015 leader boards at the end of June for relievers and starters:

Clearly the sinkerball relief specialist who targets the bottom of the zone is more common than a starting pitcher who does so, given that relievers are facing hitters only once per outing. These leaderboards are full of guys who don’t throw very hard, make their living off command, keep the ball down with sinker/slider combinations, and nibble around the outskirts of the zone where it’s harder for hitters to do real damage.

These are the types of pitchers who would suffer the most with such a change to the rulebook.

Interaction with Infield Shifts

In his book Big Data Baseball, Travis Sawchik described a synergistic combination of changes the Pittsburgh Pirates implemented to improve their run prevention. This included converting from four-seam fastballs to two-seam fastballs, which tend to sink when thrown. Intuitively, we would expect sinkers to be thrown down in the zone, with the hope that hitters will swing over top of them and beat them into the ground. The final piece of the run prevention plan was an abundance of infield shifts, so that players were situated in the areas where all these ground balls most likely would be hit.

If the bottom of the strike zone were raised, two-seam fastballs would need to be thrown higher on average, which could lead to fewer ground balls and a reduced effectiveness of defensive efficiencies gained via shifting on the infield. With the Pirates’ story as an example, I looked to see at the team level if there is a positive relationship between the number of infield shifts implemented on defense and the percentage of pitches thrown to the bottom of the current strike zone that would be contracted under such a rule change.

At first glance, I realized that infield shifting appears to be performed at very different levels between the American League and National League.

The AL has shifted 73 percent more often to date in 2015 than the NL. This is an interesting phenomenon on its own. I could see two potential reasons for this. First, the AL features the designated hitter, whom I envision as one of the hitters most likely to be shifted to the pull side. Certainly he would be shifted more regularly than an NL pitcher. Second, perhaps there is a bit of follow-the-leader happening in the AL. The Rays and Astros are the shifting kings, and I could believe as teams face division rivals and see hits taken away by cleverly shifted infielders, it may spur them to invest in shifting research as well. The more teams that come on board, the more pressure there is to keep up among direct competitors.

Looking at potential correlations between throwing to the bottom of the zone and employing infield shifts, the stories are very different between the two leagues. In the National League, the data suggest a relationship between pitching down and shifting on the infield.

The combination is especially present for the Pirates. The American League, much like in the 2015 standings, looks chaotic. Several teams that shift heavily have been targeting the expanded area of the strike zone infrequently relative to the rest of the league.

Perhaps a study with more resolution that looked at the number of infield shifts employed per pitcher or per pitch type might yield more consistent results. While certain teams may employ this combination of strategies, in general, there are many infield shifting plans used around the league.

As another related test, let’s consider potential relationships between teams that keep the ball down in general and teams that shift on the infield.

When using the percentage of pitches fewer than 21 inches above home plate rather than specifically the contracted area, the data show an even stronger relationship in the NL but still nothing at all in the AL. Really, it is a handful of AL teams that shift a lot despite not keeping the ball down — like the Blue Jays, Orioles, and Tigers — that cause the scatter plot with such few data points to become jumbled in the AL.

In any event, I suspect teams like the Pirates, Yankees and Diamondbacks, which shift reasonably higher than their league average and also throw to the low part of the zone more frequently than other teams, may have the hardest time adapting to a zone re-definition such as this one.

Summary

In parsing the commissioner’s quote about the strike zone, I have found evidence to support most of his claims. As far as imagining what effect a new rulebook strike zone definition may have on the game, my analysis has estimated that if the only change were to move the bottom of the strike zone back up from the hollow of the knee to the top of the knee, the game as a whole would resemble the 2012 game. I found this when looking at strikeout rates, walk rates and the run-scoring environment.

The most complicated aspect of predicting the outcome of making such a change is identifying particular teams or individuals who would be especially hurt by this specific strike zone change. I suggested some individual pitchers and teams that may need to make more adjustments than others if a change to the bottom of the strike zone was enforced. One other potential side effect that could be conceived at the team and individual level would be whether (usually shorter) catchers who are particularly good at framing low pitches at the bottom of the strike zone may lose some of their competitive advantage with this type of rulebook change imposed.

I have confidence in the capabilities of (most) teams’ analytics departments to adapt to a rule change like this and find new ways to gain advantages. Much like the availability of Statcast data, a change to the strike zone definition would be a new development being received by all teams at the same time, and it is up to the organizations to make best use of the data as quickly as possible to gain an upper hand.

Can you think of other potential side effects? Please leave your ideas in the comments!

References & Resources

• Special thanks to Eno Sarris for acquiring the quote from the commissioner, Brian Mills for assistance and discussing his work, and Jeff Zimmerman for assistance with access to Inside Edge data.
• Most data sources are listed beneath their respective table.
• Retrosheet, Directory of Umpires
• FanGraphs Guts!, wOBA and FIP Constants
• Baseball-Reference, League Year-By-Year Batting–Averages
• Jeff Passan, Yahoo! Sports, “Sources: MLB could alter strike zone as response to declining offense”
• Brian Mills, University of Florida/Social Science Research Network, “Expert Workers, Performance Standards, and On-the-Job Training: Evaluating Major League Baseball Umpires”
• Matthew Carruth, FanGraphs, “The Size of the Strike Zone by Count”
• Jon Roegele, Baseball Prospectus, “Baseball ProGUESTus: The Living Strike Zone”
• Jon Roegele, FanGraphs, “Early Changes to the Strike Zone”
• Jon Roegele, The Hardball Times, “The Expanded Strike Zone: It’s Baaaack…”
• Jon Roegele, The Hardball Times Baseball Annual 2014, “The Strike Zone During the PITCHf/x Era”