February 05, 2010

Rumblings about FIP

Today we had a discussion over at ACB about what should be defined as replacement level FIP. MB had just done Jeff Gray's projection and it came out to 4.32 FIP, which he claimed to be right around replacement level. This set off all sorts of alarm bells for me. I still wonder if the replacement level FIP for relievers is set too low. Let's take a look at the actual numbers and arguments and see if they disagree with me.

Here's the main argument. Starting is harder than relieving. This I agree with. The replacement level FIP for a NL starter is 5.35, which seems right to me (since FIP is more or less a proxy of ERA anyway). A quick rule of thumb for converting between the two is that a pitcher's FIP should be 1.25 x what it would be as a reliever. This is okay with me too. But given that, then a replacement level reliever's FIP should be 4.28! That seems way too low to me - in my head that's what an average reliever, not a replacement level one should do. How do the numbers bear this out?

Running the numbers, the average FIP in the National league was 4.1, which is indeed lower than 4.28. This gap doesn't seem as big as, say, the gap between an average batter wOBA and a replacement level bat, but I digress.

Where did that replacement level number come from anyway? In a series of posts on Fangraphs, Dave Cameron broke down how replacement level is defined. Basically, it was estimated that with a replacement level starter and average everything else, a team wins 38% of the time, and with a replacement level bullpen and average everything else, a team wins 47% of the time. Next, we look at the number of runs allowed in our league in a year (Dave looks at 2008, since he wrote it after the 2008 season). For example, in the 2008 American league the average FIP based on this should be 4.40, which is what a average (.500) pitcher should generate. Now, what about replacement level? Cameron cryptically says 'running the numbers through the formula gives us a 4.68 FIP'. Here's my dumb hick guess as to what he did

I'm assuming a linear approxmation here, with win% as the variable x. I'm assuming that a pitcher that has a FIP of 0 has 100% win percentage, and a pitcher that gives up 4.4 runs has a 50% win percentage. Since we have 2 data points, we have the linear approximation:

FIP = 4.4(x-1)/-0.5 = 8.8(1-x)

This a 47% reliever and a 38% starter will have

FIP_relief = 8.8(.53) = 4.66
FIP_start = 8.8(0.72) = 5.45

Which are different than what Cameron found (4.68 and 5.63, respectively). Maybe I'm doing things wrong. For one, a 0 FIP pitcher isn't going to have 100% winning percentage since he still gives up hits and walks and stuff (though he does strike a ton of guys out). But since our league average FIP is being scaled to run scoring, we'll have to do it. Maybe there's some other data point that he didn't mention that he's using for the other component of the linear approximation (or he's not using a linear model at all).

At least I learned a thing or two


David said...

b, I'm not sure if you've check this out, but this is usually where I go first to check out anything about how to calculate WAR: http://www.insidethebook.com/ee/index.php/site/comments/how_to_calculate_war/

"For pitchers, that level is set at .380 win% for starters and .470 for relievers. The exact same pitcher will perform much better in a relief role than in a starter role. So, you need different baselines. You can read the relevant chapter in The Book for that. As with nonpitchers, you need different baselines in each league: .370, .460 in the AL, and .390, .480 in the NL.

For closers, there is a closer replacement level of .570 win%. Any wins above this level gets multiplied by his Leverage Index (LI). That is, while he is not responsible for the LI he finds himself in, he is responsible for his talent that allows him to take advantage of the extra leverage (sort of like Ozzie Smith gets to play SS and reaps the benefit of all the extra opps). We are only giving credit to the closer for the leverage above the .570 win% level. Credit GuyM for this insight. "

I think The Book discusses this too. I'll have to look into that.

Berselius said...

I believe Tango's win% numbers - they mostly make sense to me, though I'm surprised that a replacement level reliever is that close to an average one in terms of impact on a game. I guess you're assuming they're only pitching 1 inning anyway. What gets me, as you can see above, is how you convert those numbers into a replacement level FIP.

One other thing that bugs me is that it's based on runs allowed. That's it. It seems to me you should do them separately and do runs allowed by starters and runs allowed by relievers.

Of course, I should just check my copy of The Book, since I own it but am still too lazy to plow through it after a year and change lol

Brad said...

Very interesting stuff. It makes me wonder about replacement level tRA because it's always so higher than FIP.