Too close to call.
It's not the satisfying answer to the Great Belichick Debate, which has seen him declared by most observers to be either an infallible genius or overzealous tinkerer, but it's the most accurate one.
Did he make the right decision?
Too close to call.
No matter how we fiddle with or adjust the probabilities to account for the game situation and the quality of the two offenses and defenses, it's difficult to find a dramatic difference between the choices of going for it or punting.
Take the first option -- the one Bill Belichick chose. Teams attempting to convert a fourth-and-2 have successfully done so at a 48.6 percent rate this season, well down from 62.3 percent last season. The "true" likelihood of such a play being converted by an average team in an average situation is closer to that latter figure, based on historical data.
Of course, the phrase "average team in an average situation" simply does not apply here. The Patriots don't have an average offense, they have the league's best DVOA; the Colts' defense ranks sixth. Although the Patriots had failed to convert in two of their three previous third-and-2 situations (including the one directly preceding the decision to go for it), their chances of making it in this situation were greater than the average team's.
On the other hand, the probability has to be adjusted for the situation. Robert Mathis had been abusing right tackle Nick Kaczur all night, meaning Brady wouldn't have time for the combination of Randy Moss and Wes Welker to run anything resembling an intricate route pattern. Dwight Freeney lurked on the other side. Furthermore, the probability of a team going for it on fourth down might very well be artificially high because of selection bias -- teams are far more inclined to go for it on fourth down against the Lions or the Rams than against the Vikings or Ravens.
Throw all these numbers into a big soup, stir them however you'd like and you'll end up with an expected conversion rate of about 60 percent. It might be 63, it might be 57; truthfully, it's not going to be enough to change our analysis.
The Colts won't score every time they get the ball on the opposition's 29-yard line, but they will score most of the time. Toss in momentum and the quality of the Colts' offense versus the Patriots' secondary, and you can estimate, say, an 85 percent chance of the Colts scoring in that situation. That makes Belichick's decision to go for it a little stronger, upping the Patriots' chances of winning by going for it to maybe 66 percent.
Then, it comes down to punting and where Manning gets the ball, which requires even more theoretical assumptions. Chris Hanson has a 39.6-yard net average, but the game was in a dome, and the Colts don't have great return units. If we just assume a 40-yard kick, the Colts get the ball on their 32-yard line with two minutes to go and one timeout. If you believe that the Colts had a 34 percent chance or better of scoring a touchdown in that situation (100 percent minus the 66 percent chance we mentioned a moment ago), Belichick was right. If you think the odds are worse than 34 percent, Belichick was wrong.
If you disagree with the expected percentages of conversion above, Mike Harris of Football Outsiders has developed a nifty calculator that lets you plug in your own averages and figure out whether Belichick made the right call by those figures. You can find that calculator here.
The key factor that the cacophony of responses seems to be missing is that you can't judge Belichick's decision by the fact that it didn't work. As we've mentioned more than once in these pages, you cannot judge decisions by their outcome. You have to consider the process that goes into them, and then decide whether they're right or wrong at the moment they're made.
Think back to another controversial Belichick decision made in the heat of a prime-time game -- his decision to take a safety on purpose down one point during the fourth quarter of a Monday night game against the Broncos. Of course, the Patriots ended up getting the ball back and won the game. Belichick took virtually no flak after the game for his unconventional choice and was instead hailed as an aggressive, brilliant game manager.
If Kevin Faulk had stumbled 2 feet forward, Belichick would be spoken about in those glowing tones today by virtually everyone now lining up to criticize him. That doesn't make his decision correct or incorrect, any more so than Faulk's coming up short does. If Belichick's decision was wrong, it was wrong from the moment the play call went to Tom Brady. And with everything we know about the situation, it's impossible to say whether that was truly the case.
Brady and Manning are two players who had big days, according to this week's Football Outsiders Quick Reads. Click here to learn more about what DYAR numbers mean and how they are computed. Note that our opponent adjustments are currently at 90 percent strength.