How can you tell which relievers have had the most impact for their teams? Some might simply say "saves," though that immediately eliminates any pitcher who hasn't been given the opportunity to pitch in the ninth inning. It also doesn't seem quite right to say that Huston Street and Tom Wilhelmsen, for example, have performed equally. Each has 11 saves, but Street has struggled with a 6.96 FIP while Wilhelmsen has been far more effective at 2.43.
Others might point to ERA, though it can be very unreliable over small amounts of innings, especially since it poorly accounts for ownership of inherited runners. We can see that with the case of Kansas City's Tim Collins, who has actually been very good this year with a 1.75 FIP and 10 scoreless outings in 13 appearances. But thanks to one poor game earlier in the month, he saw his ERA jump from 2.79 to 5.59 overnight.
So what can we use that applies equally to all relief pitchers, regardless of role, yet also takes into consideration the primary job of getting outs (or not) in the most crucial situations? For that, we turn to shutdowns, a FanGraphs stat that attempts to measure the most basic question of "did a relief pitcher help or hinder his team's chances of winning a game?"
Shutdowns (and its negative equivalent, "meltdowns") works on the premise of win probability added. You can read the full description here, but it is essentially a context-based stat that identifies how important each play in a game was towards a team winning or losing.
This works perfectly for relievers, since the setup man who enters with the bases loaded in the eighth is often much more directly involved in the outcome than the closer who gets to start a clean inning in the ninth, even though it's the latter pitcher who will get the "save." Shutdowns are awarded when a pitcher increases his team's win probability by at least 6 percent in a game, while meltdowns come when a reliever costs his team by at least that same amount.
Using shutdowns, who finds themselves on the top five list of most essential relievers this year? The answers, in some cases, may surprise you.