October 19, 2009

Quarterback statistics

When evaluating groups of quarterbacks, I tend to use the NFL's much maligned "passer rating" statistics. It synthesizes a "quarterback's completion percentage, passing yardage, touchdowns and interceptions" into one number.

There are a lot of problems with it:

- Almost nobody knows how to calculate it. Wikipedia has the formulas here. Hence, nobody knows what a good number is. The NFL average is typically in the low to mid-80s. A rule of thumb is that 100 is excellent.

- Because it includes touchdown passes v. interceptions, which are inherently small sample sizes, it's not terribly stable from year to year. For example, according to passer rating, the sixth best single season of all time was Milt Plum's in 1960 when he threw 21 touchdowns v. only 5 interceptions. Plum was a good quarterback, but that year's passer rating was anomalous. Luck plays a sizable role in a single season's touchdowns and interceptions. Nonetheless, when you aggregate over multiple seasons or across multiple players, the larger sample sizes make the emphasis on touchdown passes and interceptions more reliable. After all, there is a pretty high correlation between them and winning.

- Passer Ratings is a "Pedro Martinez statistic" in that it doesn't give credit for durability. A brilliant but fragile pitcher like Pedro Martinez looks very good in sophisticated statistics like ERA but not quite so great in simple counting statistics like career wins. Similarly, guys who played forever like Dan Marino, Brett Favre, and John Elway don't look quite as good as the guys with shorter careers like Steve Young and Jeff Garcia. (Garcia is an interesting case in that he was a career journeyman -- his statistics might be inflated because he would get plugged in when the situation was ripe for his particular talents and benched when they weren't.) Similarly, Daunte Culpepper, is ranked 11th all-time in career passer rating, but has only managed to start about 9 games per season on average.

On the other hand, passer rating has advantages over simpler qb statistics like yards passing. If you rush for 150 yards per game, you are probably better than somebody who rushes for 100 yards per game, because rushing is debilitating, so there are diminishing marginal returns. On the other hand if you pass for 300 yards per game, you aren't necessarily better than somebody who passes for 200 yards per game. Throwing a football isn't that tiring. Also, although it's less true in today's passing-oriented offenses, but in the past, the big yardages were typically wracked up by quarterbacks who fell behind early and had to mount desperate comeback attempts, often unsuccessful.

Then there are the problems with all current quarterback statistics:

- Quarterbacks are much more dependent on their receivers, running backs, and offensive lines than, say, baseball pitchers are on their teammates, so San Francisco quarterbacks throwing to Jerry Rice, for example, will rate better than guys throwing to people who weren't Jerry Rice.

- There's no home field adjustment like in modern baseball statistics. This goes to the heart of the long-running Peyton Manning vs. Tom Brady dispute. Manning usually has better passer ratings, but he plays his eight home games per season indoors, while Brady plays outdoors in suburban Boston.

Nonetheless, I think history has largely vindicated passer rating. Why? Notice that 12 of the top 15 quarterbacks of all time in career passer rating are currently active. This suggests that it measures pretty accurately the direction that improvements in offensive play are taking football in order to win games.

Other statistics have been invented. For example, Adjusted Net Yards per Pass Attempt looks like a pretty good single number stat based on yards per attempt: It dispenses with completion percentage and augments yards per attempt by adding 20 yards per touchdown pass and subtracting 45 yards per interception. It also subtracts yards lost when sacked.

This has the advantage of making the statistical differences more comprehensible than passer rating. For example, over his career, Peyton Manning's team has averaged a 7.13 yard gain everytime he passed (or was sacked), while his younger brother Eli Manning only averages 5.39 yards. So, if both brothers drop back to throw 30 passes per game, Peyton's team will gain 214 yards and Eli's team only 162 yards. I would guess that Peyton's extra 52 yards is worth somewhere between a field goal and a touchdown more points per game, plus it keeps the other team's offense off the field a little more.

Yet, the results for ANY/A are pretty much the same as for passer rating, though. Steve Young drops from #1 to #3 for career, and Peyton Manning moves up to #1 with 7.13 yards per attempt. (Young doesn't get credit for his rushing yardage, which seems a shame, but that's a tricky thing to account for because a lot of quarterbacks rush mostly on 3rd and 1 quarterback sneaks. If they average 2 yards per carry, they're doing fine, but including that would lower their overall yards per play average. I mean, you don't want to penalize the QBs who are good at sneaks worse than the ones who are no good at sneaks and thus never attempt them. On the other hand, it would be nice to find a way to credit quarterbacks who were genuine rushing threats like Young and Michael Vick.)

The top of the ANY/A list continues to be dominated by active players, with Young, Joe Montana, and Dan Marino the only old-timers in the Top 10.

Among the 29 active quarterbacks on the all-time Adjusted Net Yards per Pass Attempt career list, Culpepper is the highest ranking black quarterback at 12th with an average of 5.97 yards per attempt, Donovan McNabb is at 15th, David Garrard at 16th, Byron Leftwich at 20th, Jason Campbell at 21st, and Michael Vick at 26th (4.91). Some of these guys were good to very good runners, especially when they were younger, so they were probably a little more effective overall than this ranking suggests.

Overall, not too bad, but not at all "the future of football," as was widely hyped as recently as a few years ago.

My published articles are archived at iSteve.com -- Steve Sailer


Anonymous said...

I don't mean to take the conversation too far off-topic, but in pure statistics, the whole idea of starting off with a defining equation is a little bogus.

Instead, in pure statistics [i.e. 100% non-deterministic], you allow the statistics themselves to define the best-fit equation [e.g. via least-squares in the linear case, or Lagrange Multipliers in the non-linear case, or maybe even using some method in Fourier Analysis that I'm not familiar with].

Although, having said that, Steven Miller's derivation of Bill James's Pythagorean expectation is one of the more interesting [and readable] articles in deterministic statistics that I've ever encountered.

John Seiler said...

I know people don't like it, but shouldn't the criterion be championships? Especially for quarterback, the most dominant position of any of the major team sports. Sure, it slights Marino, Fouts, and other gunslingers. Guys like Montana and Brady sure have great coaches. And defenses are important.

But are they the top QBs because they have great coaches, or because great coaches know how to pick the top QBs? And every great quarterback is great at winning in the clutch, which is why Tarkenton and Jim Kelly don't make the list.

Here's the top quarterbacks since 1950, with the number of NFL and/or Super Bowl championships:

Starr 5 -- always underrated.
Montana 4
Bradshaw 4
Brady 3
Aikman 3
Graham 3
Unitas 3 (knocked out of last championship game, Super Bowl 5)
Layne 2 (and he was on a Lions team that won a 3rd, but broke his leg before the championship game)
Elway 2
Griese 2
Roethlisberger 2

Steve Sailer said...

You get a larger sample size for career playoff record than championships -- maybe one point for each game started and one for each win. You then have to normalize it for number of playoff games per season, which has gone up since the 1950s.

OneSTDV said...

Brady is the most overrated QB in history (well at least until 2007 when he got one of the most loaded offenses in history with Belicheck passing on almost every down).

Essentially, Brady's pre-2007 characterization as the game's best QB and an all-time great was based on his 3 championships. but if you look at his numbers, it's clear he had little part in much of those championship runs, especially his first year. It was primarily a playmaking, turnover heavy defense and expert coaching by Belicheck.

I don't feel like getting all the numbers but here's the stats link from 2001:


OAK: 0 TD, 1 INT, 1 Fumble
PIT: 115 yards, 0 TD
SB: 145 yards, 1 TD, offense scored 13 pts (defense had INT-TD)

TENN: 50% passing, 200 yards, 1 TD

There's plenty of other very mediocre performances. Until 2007, Brady's legacy as an all-time great was completely unwarranted.

Since then, I still give way more credit to Moss, Welker, and Belicheck. When Culpepper had Moss, he was equally as good as Brady is now. Now Culpepper can't even play for the Lions.

Alan Stewart said...

As you say, there has been great improvement in the sophistication and skill of offensive play. That mans that adjustment must be made to ensure that today's quarterbacks' raw numbers are DOWNGRADED so they are not overrated because of the greater skill of their teammates and coaches.

Lot easier to rack up statistics today than when you called your own plays, dropped back protected by a line that was supposed to block with their arms held out square and stiff, looking for receivers who might be lying on the ground, it being legal to bump them all the way down the field.

The best stat is something like AYPPA converted to annual percentile rankings, which could be added together (to measure career value) or averaged (to measure value in those years you were playing). Milt Plum gets 100 in his big year and we work down from there. Players are thus always compared to their competitors, their contemporaries. No advantage accrues from playing in a great offensive era.

If you believe quality of play has increased over time, you can discount previous years' numbers by whatever amount you choose.

Anonymous said...

Uhh, how about "sired the most sons who won Super Bowl MVPs"?

Steve Sailer said...

Yes, Otto Graham's passer ratings from 1946-1955 are way out beyond anybody else from his era. He ranks 16th all time, and the next highest of any contemporary of his is Norm Van Brocklin at 90th.

DCThrowback said...

One comment:
Link Duante Culpepper's career with Randy Moss and without. I think you'll see a disparity.

You can do the same with Brady's if you want to be fair, but Brady won two Super Bowls without Moss. In fact, if you look at some of the WRs and RBs Brady won titles with (guys like Reche Caldwell and Chad Jackson and pedestrian RBs like Antowain Smith), you begin to see the genius of Belicheck's system and why it was (is?) successful for so long. Also didn't hurt they put so much focus on a cohesive offensive line.

Anonymous said...

Manning has been beautiful this year. Its fun to watch the Colts play offense. Reggie Wayne and Dallas Clark as as good a reciever/tight end combo as their is. Collie looks like a bigger Wes Welker. I eagerly anticipate watching the Colts on television just to see good offense competently performed by 11 real professionals, even though they aren't "my team".

Peyton Manning has been nothing but classy his whole career. And this is an old Alabama fan saying that.

OneSTDV said...


Exactly, it's Belicheck's brilliance and a playmaking defense (look at the turnover stats for the Pats in those three title runs, they're off the charts). Brady was little more than Dilfer-esque until 2007. Moss made him just like he made Culpepper, then add in Welker whose probably a top 10 WR and Belicheck's refusal to ever run the ball, and you wonder how he puts up those numbers. Oh and add in Donte Stallworth in 2007, a very good TE whose name I forget, and a pretty good RB in Larence Maroney and possibly the best screen game with Kevin Faulk and maybe the best OLine in the game. Put Jim Sorgi back there and he'd throw for 30 TD's.

Belicheck is the all time great. Brady is not. It's a shame that so many put Peyton below Brady when Peyton is leaps and bounds above him.

[NOT a Colts fan either]

Truth said...

You could make a strong case that Moss is the greatest non-quarterback who ever played (although in my opinion, Reggie White, Lawrence Taylor and Jim Brown would have to be considered.)

The QBs he's played with:

Brad Johnson
Kerry Collins
Andrew Walter
Tom Brady
Matt Cassell

QBs who've had their career year with Moss:

Kerry Collins
Andrew Walter
Tom Brady
Matt Cassel

Moss should have won the Heisman as he was clearly the best player in America both his freshman and sophomore years at Marshall. He started off at Florida state and redshirted because of off the field issues. At the time, Bobby Bowden had Moss and Deon Sanders, maybe the two fastest guys ever in the NFL checking each other as 18 year olds...things have sure changed for Bobby.

By the way he and Lou Holtz both said that he was the best high school player they ever saw.

Anonymous said...

Steve, check out footballoutsiders.com, they do great stat analysis work for football. (http://en.wikipedia.org/wiki/Football_Outsiders )

Steve Sailer said...

Do top wide receivers like Randy Moss take breathers on the sidelines enough to calculate on a play by play basis how much they individually add to the team?

I remember years ago the Chicago Bears having a quarterback controversy, but as far as I could tell, the thing that seemed to matter in terms of the effectiveness of the overall offense wasn't who was the quarterback but whether or not receiver Curtis Conway was in the game and in full health?

wake up said...

classic steve sailer analysis once again missing the forest for the trees....qb ratings? for the real world all that matters is resilient leadership under duress which produces victories that actually matter (playoffs/championships vs regular season) and top performance when important games are actually on the line i.e. coming through in high pressure situations...that is the stuff that qb greatness is made of in the real world.....and everyone knows it except the stat geeks......

on the other hand qb ratings sorted by statistic geeks sitting in their underpants in front of computers yield bizarre results like daunte culpepper and philips rivers ranked as all time greats....yeah! betcha didn't know that!

steve you always show a bias for consistent robotic output in human endeavor....have you noticed that tendency in yourself? which dovetails perfectly with your religious faith in the national testing regime results as a barometer of our brain power....if it was up to professor steve sailer it's likely that both young thomas edison and young albert einstein would've been put on a track for careers in "ditch-digging" due to erratic output and general unwillingness to jump through the hoops.......

hey steve isn't the greatest songwriter of all time that warren lady because doesn't she have the most charted billboard entries in history? is it her or is it the senator hatch from utah who has published thousands of songs? which is it steve? who's the greatest?

Steve Sailer said...

I have a bias in favor of large sample sizes as being more reliable than small sample sizes.

eh said...

Speaking of statistics, or a lack thereof,...

More ghetto-like violence and dysfunctionality brought to a college campus via recruitment of players who, in many, many cases, have no business being there in the first place

And what share of that, as well as the other missteps of high profile athletes we hear about all the time, is due to black players?

Albeit this poor guy (such a sad, needless death) did not stab anyone. But 'being in the wrong place at the wrong time' seems to be a bigger problem for some people than for others. Hint hint.

...19-year-old Brian Parker of Sarasota, Fla., who also was stabbed but sustained minor injuries. The sophomore wide receiver, who is academically ineligible to play this season,...

How tough is the course load of your average football player?


About to become a father, with no visible means of support. How could there be while supposedly being a full-time student and football player?

Yes, white QBs do, in general, seem to outperform black QBs. And, again generally, Whites seem more likely to possess the skill set that will lead them to become a QB. And Limbaugh was right: the media seems to ballyhoo the achievements of black QBs, probably in an attempt to compensate for all the above.

And that gets some attention, i.e. the explicit race angle.

But what seems to get no attention is the growing amount of violence and trouble on college campuses (and overall in sports), due predominantly to black athletes. Which you never hear stated openly.

Anonymous said...

On a tangential note, seeing the name Daunte Culpepper, pronouced "Dante Culpepper", reminds me that I'm getting slightly annoyed at the ridiculous spectacle of sports announcers having to present NAM athletes with misspelled names. Current examples include the Bears' Danieal Manning (pronounced, apparently, like "Danielle Manning") and a pitcher on the White Sox named Jhonny Nuñez, who is not the only "Jhonny" in the game.

Peter A said...

it's clear [Brady] had little part in much of those championship runs, especially his first year

This statement displays such mind boggling ignorance about football that you should be banned permanently from posting here OneSTDV. I'm not kidding. That perhaps the single stupidest, most indefensible comment I've ever read here. On a par with the earth is flat or a duck is a fish. If I were your father I would disown you for making that comment in public.

josh said...


Have you ever been to Footballoutsiders.com it is a sister site to Baseball Prospectus. They use play by play data to compare each offensive play to league average accounting for down, distance to first down, and position on the field. A quarterback is judged on plays when he drops back to pass; so this accounts for fumbles and rushing plays and it gives a weighting to yards, touchdowns, interceptions, etc. that actually has something to do with how important they are in winning a football game. They give the scores as both a rate stat, percentage above of or below average, and a total stat, yards above replacement. They even do adjusted rates and yards accounting for the quality of opposing defenses faced.

It's the closest thing I've found to decent Sabermetrics for football.

By the way, since 1999 Payton Manning has been the number one quarterback in the league 6 times , number two twice, and has never finished outside of the top five. He's probably the best of all time.

josh said...

"Bobby Bowden had Moss and Deon Sanders, maybe the two fastest guys ever in the NFL"

Darrell Green and Bob Hayes disagree.

Anonymous said...

I don't think one can take anything away from Brady, but one could say that Belicheck picked him for a reason and give credit to the system. Most stars will end up with a system built around them if one doesn't already exist, however.

Truth said...


No, that's what they hire agents for.

"for the real world all that matters is resilient leadership under duress which produces victories that actually matter"

Then I guess Trent Dilfer is a great QB.

rast said...

Football outsiders, Steve. They've already considered all this stuff in far more detail.

Barring that, go with YPA (yards per pass attempt), adjusted for sacks and turnovers.

keypusher said...

I am sure this point has been made, but passing is so critical in the modern NFL that a guy who can stand still, scan the field and hit the open receiver is much more valuable than a guy who can run. If we went back to 1970s rules, when winning teams would run three times or even six times as much as they threw, a running QB would be much more valuable. Obviously the Miami Dolphins could use one now.

Anonymous said...

Let me be a second vote for Football Outsiders. If you're interested in statistical analysis of football, it is a must-visit site.

Marketing said...

QB Rating is just a single stat, third down conversions are more likely to value a good QB performance in big games. Great article!.

Can you please send me your ad rates for isteve.blogspot.com to angel{at}hstreetmedia.com

not a hacker said...

Steve, a pitcher who threw over 200innings in 7 different seasons cannot be called "fragile."

Bob Montgomery said...

I think the current players' domination of the ANY/A list is due in part to the exclusion of QBs who had careers mostly before 1969 (like Bart Starr), when they started counting sacks, and also due to the short (incomplete) careers of a lot of those players.

If you eliminate all the current players under the age of 30, then the lists are pretty balanced between active and retired players:

1. Peyton Manning
2. Steve Young
3. Kurt Warner
4. Joe Montana
Tom Brady
6. Dan Marino
7. Trent Green
Jeff Garcia
8. Chad Pennington
9. Duante Culpepper

The AY/A list, which includes pre-1969 careers:
1. Steve Young
2. Peyton Manning
Kurt Warner
4. Joe Montana
5. Tom Brady
6. Trent Green
7. Duante Culpepper
8. Roger Staubach
9. Dan Marino
10. Chad Pennington

Bart Starr would be at #11.

We're still missing the real old-timers like Otto Graham; not sure why they aren't included on the list.

Steve Sailer said...

"Steve, a pitcher who threw over 200innings in 7 different seasons cannot be called "fragile.""

Well, barring a long string of 20 win seasons from now through age 42, Pedro Martinez isn't exactly Warren Spahn, now is he?

DCThrowback said...

@wakeup -
Please, add a less value added comment next time. If statistical analysis is working, 95% of the time in confirms what you already know. In this case, Steve is looking for some way to measure the rankings of great QBs. In this case, he's done a pretty good job, especially for a non-sports site. Football Outsiders should probably do this, but I can understand their desire to not get too involved in the Limbaugh broo-ha-ha. I am sure the last thing Aaron Schatz wants to prove is Limbaugh was right, even though that day in the studio no one said a word about it because...it was pretty much the truth. Notice the fans in Philly have never given McNabb as much slack as national sportswriters. Are the fans racist (Well, it is Philly...), or maybe they know something because they watch the team much closer than everyone else does?

@STDV & @Truth
You guys hit the nail on the head. If anything shows, truly, what a dysfunctional organization the Raiders are, it's acquiring the best WR in the game in Randy Moss and single-handedly making him disinterested in catching footballs...and then trading him to the best team in the league for a 4th round pick. Yikes. What makes the NFL so great, though, is the fact that Pats haven't won a Super Bowl going on 5 years. It's a tough thing to do - which is why winning 3 in 4 years is something to really be lauded.

Have to agree about Manning. There's a reason he's paid $32M/year. His professionalism and skill really makes the Colts a fun team to watch...year in and year out. Also, much props to the Colts GM, Bill Polian. Polian also built the Panthers '95 team that went to the NFC Championship (in only its 2nd year) and the Bills' squads that went to 4 Super Bowls. He acquires a competitive advantage and works like hell to surround it with cost effective talent in effort to be competitive every year. Truly one of the best.

Anonymous said...

WakeUp's tone is off-putting but he does make a strong point. Sometimes, quants have a tendency to put too much faith in statistics. I think the best example of this would be the performance of Kyle Orton this season. Using any of the statistics that you cite in your post as a guide, an observer would have called Josh McDaniels crazy for choosing Orton over Cutler. However, Orton has exceeded everyone's expectations and looks like a solid Top 10 QB. That is not to say that Cutler would not have succeeded under McDaniels system but there is a tendency for quants to not appreciate that sometimes stats can be misleading when they are not considered within a specific context. According to McDaniels he could "sense" in the study of game film that Orton was a cerebral QB. Sometimes you have to beleive your lying eyes.

Anonymous said...


If you look at OneSTDV's blog he is normally very clear headed, but when it comes to Brady he seems blinded by mindless prejudice. Also, surprisingly for a guy who named his blog after the 1 S.D. gap in IQ between blacks and whites, he seems to have a fetish for double-digit IQ black QBs like McNabb (Wonderlic 14/IQ 88). He thinks McNabb is one of the all time greats.


As a Philadelphian, I can assure you that this is the case. McNabb can't read defenses, make adjustments, or consistently throw accurately over the middle. (And apparently, as we saw this weekend, keep track of how many timeouts he's called.) His whole career has been based on a great defense, an o-line that in most years gives him enough time to write a Tolstoy novel while dropping back, and has had great receiving RBs (Watters, Garner, Staley, Westbrok, etc.) over the course of his career that catch his little dink passes out of the backfield and make decent gains. He couldn't run an offense that featured a lot of down field passing.

DCThrowback said...

Of course Orton is cerebral! He went to Purdue, is known informally as the "Neckbeard" and drinks Jack Daniels like a fish. Google "neckbeard, deadspin" and have yourself some good old laughs.

Winner! Meanwhile, Jay Cutler only went to Vanderbilt. Josh McDaniel could tell right away that he was a total moron - all he had to do was look at those silly 'bama bangs.

Anonymous said...

Truth said:


Sanders last year at FSU was 1988. Moss was there in 1995.

But, what are facts, when you have Truth.

eh said...

It's surprising that no one has mentioned Doug Williams, the only black QB to win the Super Bowl. He was named MVP of the game too.

Truth said...

"Sanders last year at FSU was 1988. Moss was there in 1995."

My god you are right, you got me. The story is that Sanders came back as a pro and participated in spring drills.

Good one chief.

Truth said...

Williams, or Steve McNair, who came within 1.5 yards of winning a Superbowl.

DCThrowback said...

Williams was a fine journeyman QB who was able to parlay his competence into a job with a pretty good team that executed excellently. He is famous for that one game and little else. But at the time I can only remember the media fellation at the fact he was a black guy playing QB. I also recall there was a famous question asked during the media day (something like, "What's it like to be a black QB?") that only led to some humorousness and further cemented in people's minds the cluelessness of the MiddleAgedWhiteSportsMedia. (MAWSM).

jody said...

an important thing to know about quarterback rating is that when the formula was written, it was designed so that the measured average came out to 80. yet over time, the measured average has been increasing. it is now around 85 or 86.

so there is a flynn effect for quarterback rating. whether it is a real effect, or something else, is probably not something people will easily agree on. but my opinion is that it is a real effect. it seems obvious to me that the average NFL quarterback is dramatically better, at everything, than the average NFL quarterback only 20 years ago. football passed baseball as the number 1 sport in the US over 15 years ago. we're now at the point where both NFL football and NCAA football are bigger than MLB baseball.

so if the defensive players and defensive schemes have also improved, then that means the improvement in throwing in the NFL has probably outpaced the improvement in pass defense. again, some people will argue that the improvement in measured rating is due entirely to restricting safeties and corners from mugging receivers and tight ends. but i'm not too confident that accounts for all of the difference. i think the throwing today is simply better. quarterback is the most highly selected job in american sports.

but, the throwing is probably even better than that, because now there are 32 teams, instead of 24, as there were around the time the formula was devised. that's an additional 8 starter positions that have to be filled. so the 6 point improvement would probably be a 10 point improvement if the worst 8 starters were ignored. the talent pool of football throwers is REALLY deep today.

there are other things to account for, like, with the modern NFL's near religious focus on speed above all else, there are now dozens of black receivers who are better at running fast in a straight line than they are at catching passes that hit them right in the hands. there are more dropped passes now than ever. in the last 2 games, the titans and browns receivers dropped literally every pass that connected. i've never seen receiver play that bad.