A New Way To Measure Relief Pitchers: SOEFA

Clip art illustration of a Cartoon Tiger with a Missing Tooth

I’ve long been a critic of the save statistic, and I don’t need to rehash why it’s the scourge of the baseball world, but relief pitching is still an important part of the game and we often struggle to properly measure it. Won/Loss record and saves tell you nothing about a player’s individual skill, especially not relievers, and even things like ERA don’t do a lot of good because relievers aren’t charged for runners they let in belonging to another pitcher and can get charged with runs allowed by the pitchers who come after.

Strikeouts, walks, and homeruns allowed (the basis of FIP) are good measurements, but FIP inherently strips away context. And context does matter for relief pitchers. It’s an elite reliever’s job to come in and strand runners, so strikeouts are good and homeruns are bad, but sequencing is really important and it matters a lot when they get outs and when they allow baserunners.

In a sense, FIP and similar statistics are good, but they aren’t perfect because they’re context neutral and we might want some context in reliever stats. Win Probability Added (WPA) is a typical way to fix this, but this feels too context dependent for me. WAR is always a nice combination of these kinds of measures, but WAR is a counting stat so how much a reliever is used matters a lot, and relievers are often used incorrectly.

My point here is not that I can come up with something better, but rather that I want to try to add something. I always look at reliever stats and find logical holes more often than with position players and starters. I want a reliever stat that measures context, considers the peripheral numbers, and also understands the luck involved. I didn’t find one out there that satisfied me, so I went to work inventing one.

I’ll say this. This isn’t perfect and I want to improve it. Flaws you may find in the method should not cause you to discount it, but rather to add to the discussion. This is a first crack. I hope you find it useful.

The Goal

So first, I started with a question: What is the job of a relief pitcher? Here was my answer:

  1. Strand runners
  2. Don’t allow baserunners
  3. If you allow baserunners, don’t let them score.

With that outlined, I went to work thinking about how to measure each and came up with the following statistic that I will call SOEFA, pronounced like “sofa.” It stands for Strand On-base ERA FIP Average and should be thought of as a way to measure relievers from your sofa. Yes, I have a whimsical side.

It has four components, let’s walk through them.

The Formula

First is Strand Rate+, which I calculated as what percent better or worse a reliever is from league average at stranding runners. League average is 70%, so if you strand 100% of your inherited runners, your Strand Rate+ is .43 because you are 43% better than league average.

Second, is your Expected OBP+ or xOBP+ which is your opponents on base percentage calculated as a percentage deviation from league average just like SR+, except that I regress your hits allowed based on league average BABIP because sometimes batters get lucky hits.

Third, is my version or ERA+, which is just like normal ERA- except I scale mine to zero instead of 100 like the major stat sites and invert it. Same principles regarding deviation from average applies. FIP+ is exactly the same, except I use FIP-. These numbers are park adjusted.

To combine them, I add SR+ to xOBP+ and then add ERA+ to get eSOEFA. I then repeat the same process and replace ERA+ with FIP+ to get fSOEFA. A pitcher’s SOEFA score is the average between the two.

The output gives you a number that sets league average at zero and ranges technically from negative infinity to about 2.5, but generally speaking you won’t see a reliever fall below -2.5. Basically it’s a -3 to 3 scale that puts good relievers on the plus side and bad ones on the negative side.

Additionally, at my discretion, relievers who have inherited fewer than five baserunners during the season (this number will likely be fluid based on where we are in the season) are given a league average SR+ so that if you don’t ever inherit runners you aren’t unfairly punished because you are not given sufficient opportunity to strand them or you are not given credit for an awesome strand rate if you strand the only runner you inherit.

I’m pretty happy with the first round of results. The first run of results came from stats entering June 25th and it generally lines up with my impression of the best performing relief pitchers in baseball. I have no idea if this stat is predictive or how long it takes to stabilize. Right now, it correlates with ERA and FIP at -.73 and -.75 despite the fact that each is only 1/6 of the input and the R squared is around .6 using it to predict FIP, if those kinds of things interest you.

It’s experimental. It’s meant to be fun and maybe helpful.

A word of note is that Fangraphs and B-R seem to use different cutoffs for which relievers “qualify,” so this output may be missing a few relievers. I’m sorry about that. The great thing about this statistic is that I can easily produce the number for any reliever in baseball in less than two minutes. If you want to know how a reliever measures up or how a reliever did during a given season, just ask and I can provide the number based on a handy program I wrote. Hit me on Twitter @NeilWeinberg44 and I’d be happy to provide the SOEFA for any reliever.

Thanks for reading and I welcome any feedback. Who knows, maybe this will work.

Below are the SOEFA for the vast majority of qualifying relievers up through 6/24/13. If you want to know the SOEFA of a reliever not on this list or would like an update score, please let me know.

Rank Player Team SOEFA
1 Sergio Romo Giants 0.99
2 Jason Grilli Pirates 0.95
3 Junichi Tazawa Red Sox 0.92
4 Kevin Gregg Cubs 0.92
5 Drew Smyly Tigers 0.9
6 Joaquin Benoit Tigers 0.89
7 Jordan Walden Braves 0.88
8 Robbie Ross Rangers 0.87
9 Mark Melancon Pirates 0.85
10 Jesse Crain White Sox 0.83
11 Edward Mujica Cardinals 0.79
12 Brett Cecil Blue Jays 0.79
13 Greg Holland Royals 0.75
14 Oliver Perez Mariners 0.74
15 Trevor Rosenthal Cardinals 0.74
16 Kenley Jansen Dodgers 0.72
17 Glen Perkins Twins 0.71
18 Koji Uehara Red Sox 0.7
19 Preston Claiborne Yankees 0.69
20 Sam LeCure Reds 0.68
21 Casey Janssen Blue Jays 0.64
22 Mariano Rivera Yankees 0.63
23 Luke Gregerson Padres 0.62
24 Craig Kimbrel Braves 0.62
25 Sean Doolittle Athletics 0.6
26 Edgmer Escalona Rockies 0.56
27 Tommy Hunter Orioles 0.56
28 Brad Ziegler Diamondbacks 0.54
29 Joe Nathan Rangers 0.53
30 Joe Smith Indians 0.53
31 Vin Mazzaro Pirates 0.51
32 Jim Henderson Brewers 0.5
33 James Russell Cubs 0.49
34 Casey Fien Twins 0.48
35 Tim Collins Royals 0.47
36 Shawn Kelley Yankees 0.47
37 Brian Matusz Orioles 0.46
38 Addison Reed White Sox 0.46
39 Tanner Scheppers Rangers 0.45
40 Rafael Soriano Nationals 0.44
41 Aroldis Chapman Reds 0.44
42 Joel Peralta Rays 0.43
43 Matt Reynolds Diamondbacks 0.43
44 Brandon Kintzler Brewers 0.43
45 Ryan Cook Athletics 0.42
46 Chad Qualls Marlins 0.42
47 Cody Allen Indians 0.4
48 Andrew Miller Red Sox 0.4
49 David Robertson Yankees 0.38
50 Seth Maness Cardinals 0.36
51 Bobby Parnell Mets 0.36
52 Matt Belisle Rockies 0.36
53 Josh Outman Rockies 0.36
54 Rex Brothers Rockies 0.35
55 Jonathan Papelbon Phillies 0.35
56 Dale Thayer Padres 0.35
57 Darren O’Day Orioles 0.33
58 Justin Wilson Pirates 0.33
59 Luke Hochevar Royals 0.31
60 Grant Balfour Athletics 0.3
61 John Axford Brewers 0.29
62 Ernesto Frieri Angels 0.29
63 Drew Storen Nationals 0.27
64 Bryan Shaw Indians 0.26
65 Nate Jones White Sox 0.26
66 Luis Avilan Braves 0.25
67 Anthony Varvaro Braves 0.25
68 Anthony Swarzak Twins 0.24
69 Paco Rodriguez Dodgers 0.24
70 Jean Machi Giants 0.2
71 Tyler Clippard Nationals 0.19
72 Matt Thornton White Sox 0.19
73 Steve Delabar Blue Jays 0.18
74 Craig Stammen Nationals 0.17
75 Tony Watson Pirates 0.17
76 Pat Neshek Athletics 0.16
77 Jamey Wright Rays 0.16
78 J.P. Howell Dodgers 0.16
79 Cesar Ramos Rays 0.15
80 Alfredo Simon Reds 0.15
81 Troy Patton Orioles 0.15
82 Matt Lindstrom White Sox 0.14
83 Jim Johnson Orioles 0.12
84 Carter Capps Mariners 0.11
85 Ryan Pressly Twins 0.11
86 Steve Cishek Marlins 0.11
87 Darin Downs Tigers 0.1
88 Antonio Bastardo Phillies 0.09
89 Charlie Furbush Mariners 0.07
90 Brian Duensing Twins 0.07
91 Yoervis Medina Mariners 0.07
92 Jerry Blevins Athletics 0.07
93 Tom Gorzelanny Brewers 0.06
94 Jared Burton Twins 0.05
95 Jose Veras Astros 0.05
96 Joe Kelly Cardinals 0.05
97 David Hernandez Diamondbacks 0.04
98 Ryan Webb Marlins 0.04
99 Aaron Loup Blue Jays 0.03
100 Wesley Wright Astros 0.01
101 Bryan Morris Pirates 0.01
102 Burke Badenhop Brewers 0
103 Dane de la Rosa Angels -0.02
104 Adam Ottavino Rockies -0.04
105 LaTroy Hawkins Mets -0.04
106 Cory Gearrin Braves -0.06
107 Joe Ortiz Rangers -0.08
108 Wilton Lopez Rockies -0.08
109 Brandon Lyon Mets -0.08
110 J.J. Hoover Reds -0.08
111 Mike Dunn Marlins -0.09
112 Fernando Rodney Rays -0.1
113 Hector Ambriz Astros -0.1
114 Paul Clemens Astros -0.13
115 Tom Wilhelmsen Mariners -0.13
116 Matt Guerrier Dodgers -0.13
117 Josh Roenicke Twins -0.17
118 Jose Mijares Giants -0.21
119 Michael Gonzalez Brewers -0.23
120 Jonathan Broxton Reds -0.25
121 Jake McGee Rays -0.25
122 Matt Albers Indians -0.26
123 A.J. Ramos Marlins -0.26
124 Scott Rice Mets -0.29
125 Nick Hagadone Indians -0.31
126 Travis Blackley Astros -0.33
127 Vinnie Pestano Indians -0.34
128 George Kontos Giants -0.35
129 Mike Adams Phillies -0.39
130 Clayton Mortensen Red Sox -0.4
131 Garrett Richards Angels -0.43
132 Heath Bell Diamondbacks -0.46
133 Esmil Rogers Blue Jays -0.5
134 Ronald Belisario Dodgers -0.51
135 Jeremy Affeldt Giants -0.55
136 Brandon League Dodgers -0.55
137 Jeremy Horst Phillies -0.58
138 Kelvin Herrera Royals -0.67
139 Carlos Marmol Cubs -0.72
140 Huston Street Padres -0.82
141 Anthony Bass Padres -0.94
142 Hector Rondon Cubs -1.24

 

Advertisements

20 responses

  1. […] he has been one of the better relievers in all of baseball this year. (New English D’s new SOEFA rankings put him at 5th entering the day!) After last night’s ugly one, this one was a more […]

  2. […] do look at SOEFA, the new reliever stat from New English […]

  3. […] 2013 season is better than anything he’s ever done, our new reliever statistic SOEFA, and why RBI and Wins aren’t good statistics for measuring individual performance. The author […]

  4. […] week we introduced are very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines strand rate, expected OBP against, ERA-, and FIP- into a […]

  5. […] week we introduced are very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines strand rate, expected OBP against, ERA-, and FIP- into a […]

  6. […] year we introduced are very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines strand rate, expected OBP against, ERA-, and FIP- into a […]

  7. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines strand rate, expected OBP against, ERA-, and FIP- into a […]

  8. This is really neat. Looking forward to more on this.

  9. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines strand rate, expected OBP against, ERA-, and FIP- into a […]

  10. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines inherited runner strand rate, expected OBP against, ERA-, and […]

  11. […] marks are 9th, 14th, and 15th among qualified MLB relievers this year. By our own reliever stat, SOEFA, he ranks 7th among pitchers with at least 20 innings pitched. By all accounts, he’s having a […]

  12. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines inherited runner strand rate, expected OBP against, ERA-, and […]

  13. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines inherited runner strand rate, expected OBP against, ERA-, and […]

  14. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines inherited runner strand rate, expected OBP against, ERA-, and […]

  15. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines inherited runner strand rate, expected OBP against, ERA-, and […]

  16. […] New English D’s own propriety metric, SOEFA, each of these four grade out as above average relievers for the season with Benoit and Smyly being […]

  17. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines inherited runner strand rate, expected OBP against, ERA-, and […]

  18. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines inherited runner strand rate, expected OBP against, ERA-, and […]

  19. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines inherited runner strand rate, expected OBP against, ERA-, and […]

  20. […] year we introduced our very own reliever rankings called SOEFA, which you can read about in detail here. For a brief refresher, it combines inherited runner strand rate, expected OBP against, ERA-, and […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: