2000: Chad Pennington, Pick 18, Career Rating: 89.7; Marc Bulger, pick 168, Career Rating: 85.7, Tom Brady, Pick 199, Career Rating: 92.9
2001: Mike Vick, Pick 1, Career Rating: 75.7; Drew Brees, Pick 32, Career Rating: 89.1
2002: David Carr, Pick 1, Career Rating: 74.6; David Garrard, Pick 108, Career Rating: 85.4
2003: Carson Palmer, Pick 1, Career Rating: 88.9; Tony Romo, undrafted, Career Rating: 96.6
2004: Eli Manning, Pick 1, Career Rating: 76.4; Big Ben, Pick 11, Career Rating: 89.9; JP Losman, Pick 22, Career Rating: 76.9
2005: Alex Smith - Noted; Aaron Rodgers, Pick 24, Career Rating, 89.8; Jason Campbell, Pick 25, Career Rating: 80.5; Matt Cassell, Pick 230, Career Rating 84.2
2006: Vince Young, Pick 3, Career Rating: 68.6; Jay Cutler, Pick 11, Career Rating: 88.6
2008: Matt Ryan, Pick 3, Career Rating: 92.0 - very good - so far
So, of the last nine years, the top QB taken has turned out to be the best QB of that year, maybe once. Russell just isn't good. He won't be starting for long. And Ryan, it's too soon to really tell. Meanwhile, QBs taken, sometimes very, later have often been as good, if not better, than the top QB's in 6 of the last nine drafts you cite. There are just as many guys like Brady, Bulger, Garrard, Romo, and Cassell (picked well behind "names") as there are top picks that have panned out.
You are cherrypicking even worse. Just because the highest rated QB was not the top QB does not mean the top pick is not the best. You should at least consider all the terrible quarterbacks picked in the later rounds.
Plus, the top pick always goes to the worst team in the league the previous year. I am more impressed that Carson Palmer has managed an 88.9 rating with the Bengals than I am than that Big Ben has managed an 89.9 with the Steelers.
There are 10-15 QBs taken every year out of 250 picks. The question is whether where they were picked correlates with future success. I see no evidence to support that correlation.
Otherwise, we're taking about a very limited skill set (i.e. it translates to nowhere else on the field). The reason so many QBs are picked high isn't because their talent deserves it. It's because teams overvalue the position AND there are few QBs deemed worthy enough to even be considered.
New England is actually an interesting case study of what it means to take a QB who may not have the typical skillset, but if you place them in the right environment, they can be a true star. The point is: Their success argues strongly against the role of a high pick for the position.
Plus, the top pick always goes to the worst team in the league the previous year.
This is key. Growing up in Tampa, I watched a lot of high draft picks have great careers after they left the hell-hole they got drafted into. I've always felt that getting picked as a QB in the late first round would be best, because you could be on a good team, rather than start right away and get pummeled in the 0.003 seconds you have in the pocket.
I didn't cherrypick, I simply pointed out that the QBs scouts rate the best generally perform very well. Every now and then a Tom Brady or Kurt Warner sneaks up out of nowhere, but for the most part almost every starting QB was first round drafted and often first overall.
Also, don't confuse best passer rating for best quarterback. All sorts of things factor into the QB's passer rating, such as the quality of his receivers and offensive line, his team's style, etc. Those numbers aren't very meaningful.
Nobody would ever suggest that scouts could get it perfectly, or near perfectly, in order, but they do a pretty fantastic job. You can cherrypick data to show a few black swans, but for the most part they're highly accurate.
1990 - jeff george - fair?
1991 - dan mcguire - BAD
1992 - david klingler - BAD
1993 - bledsoe - GOOD
1994 - shuler - BAD
1995 - MCNAIR - GOOD
1996 - Tony Banks - BAD
1997 - Jim Druckenmiller - BAD
1998 - Manning - GOOD
1999 - Tim Couch - BAD
At best I would say the #1 qb pick has been 50/50 historically.
But it's not "every now and then". Look at the names of "nobodies":
Brady, Bulger, Garrard, Romo, and Cassell
And that's just the last nine years. They're as good, if not better, than "top" picks. Then you have the more historical guys like Wagner, Young, and Montana.
Picking for the QB position really does seem stochastic.
QB is not stochastic at all. These are all quarterbacks who arrived at a team that was already good. "Quarterback rating" really measures the goodness of the whole passing offense. It is no surprise that the Patriots, Rams (before this year at least), Jaguars, Cowboys, and Patriots again can give an average QB great stats.
Now, you're arguing with yourself. If it's the "system" that's getting "rated", then there's even less reason to assume a team knows what they're doing when they're picking a specific QB.
And that's the interesting bit of logical jiu-jitsu going on here. The teams picking first are the ones that are usually the worst to begin with. Selecting higher just doesn't help them to improve their lot when it comes to the QB position.
I think you misunderstand, because I'm certainly not arguing with me. I'm arguing with your methodology. Scouts do a solid job of picking the QBs that are the best.
Passer rating does not measure QBs skill accurately, therefore scouts may not pick the QBs with the best passer rating. That's because their job is to pick the best quarterback, not the one who has the highest number in some goofy equation that takes many other things into account.
QB rating is bunk. Chad Pennington has a better career rating than Dan Marino. Dan Marino is an all time great, Pennington might just barely crack the top 10 current starters.
So there's a correlation with where a QB was picked and their performance? But you can't tell me how to, objectively, judge their performance? What, we're back to relying on the teams and how they dole out playing time?
I see little evidence suggesting that where a QB was picked correlates with their skill at the position. You named the last top QBs picked in each draft year and gave the most positive assessment possible of them. I pointed out that for almost all of those years there were better QBs drafted below, and many times far below, that consensus best choice. That doesn't give me great confidence in the scouting of the position. You can't say teams do a "a solid job of picking the QBs that are the best" if every year they tend to miss the best QB available, and oftentimes by wide margins.
Now we're arguing about the role of a "system" in producing performance numbers for a QB. Again, if that's the case, then the individual QB means even less than your original assessment, where the highest weight was placed on the highest picks. If anything, there's even more reason to distrust the team's picking highest (and their scouts) because they wouldn't be in that position if their system was better. If you say a team's system, already in place, will predict a QB's future numbers (your critique of passer rating), then you're also devaluing the importance of a high pick.
The real problem is that teams weight the position too highly AND create a dearth of talent based on their selection process and the limited skills/pedigree they value. That leads to a Wal-Mart stampede in the first round, doling out big bucks, to players that won't a change a team's fortunes one way or another. The position is too dependent on many other positions.
I see you're from Browns territory. How many swings have they had at the "best" QBs in a given draft and how has that worked out for them? If anything, they've continued to make the mistake of valuing the position too highly to the detriment of the rest of the team.
Funny though that you're touting Marino's greatness. He slipped because scouts devalued his "head" and his greatness is only indicated by his raw numbers. A better example for your case, would be someone like Bradshaw or Aikman. Both were the top overall picks in their drafts and both took poor teams and eventually led them to dynasties. Then we have guys like Brady and Montana...and we're back to where we started.
Looking at your more complete list, I see a bunch of QBs who were first picked who did well, with only one or two exceptions. I see some QBs who were picked second or third who also did well. (Overall picks aren't really fair, as teams have many reasons for selecting other positions.) And I see a couple black swans, like Romo.
That's what you would expect if they were really good at their job. If they were bad at it, you'd expect more first picked QBs sucking, and more people coming out of nowhere.
Objectively measuring is impossible. Passer rating is more an indication of the quality of a team's offensive line, receivers, etc. than QB skill. It's a nice number for commentators to throw out, but it's of little use to a statistician.
I'd say maybe Pro Bowls, but again, there are other factors in play there too.
You are ignoring the huge difference between "teams are choosing the optimal QB" and "teams make decisions no better than random". The decision is neither perfect nor random.
This is getting far afield of the original critique. The point there was that scouts do a good job in selecting QBs, especially for picking the best QB (first!) in a given draft. There is very little evidence to support that claim.
Surely, selecting the top college QBs in the country isn't random. But selecting among that echelon appears to be, especially for the transition to the NFL game.
No, the fact that by the time they're eligible for the draft, and of the 10-15 taken every year, there's usually little to differentiate the top picks from the worst. What separates them is anyone's guess. That's Gladwell's point.
But if you look at the top picks and the bottom picks, you find almost none of the bottom picks starting in the nfl (you named a few who did it in the last decade) and a ton of the top picks who are, or are at least second strings to superstars. I would say that means that what separates them is far from anyone's guess. There's certainly some variance there (see Tim Couch) but it's not unpredictable.
The complicating factor, of course, is all those former first rounders but second stringers, have had significant money invested in them. It's a lot easier to cut the lower picked players but not necessarily because of their "talent".
Matt Cassell = perfect example.
Does he even get a chance on any other team? Ever?
Gotta correct you on Young. He was a promising starter who many, including myself, thought could make the Pro Bowl very soon (especially if he played with wide receivers). Then he became a head case and lost his job to a mediocre Kerry Collins.
Also, I think you should include the second and third quarterbacks taken to round out your stats...that might be interesting, with stars like Ryan Leaf :^)
Out of the 9 years listed it really only happened 3 times that a better QB was left - 2000 (Tom Brady), 2002 (David Garrard - debatable, but he's starting and Carr isn't), and 2005 (Aaron Rodgers). So 6 times out of 9 the best one was taken first - that seems pretty good.
If you go back even further I'd imagine this trend would continue - even Gladwell's main example (Ryan Leaf) was taken second to the best QB playing today (Manning).
Yea, I don't want to get into a huge debate about this either, and the names you mentioned are definitely in the same class as the first picks. My main thought was that while its not a science, the scouts are definitely not just randomly picking players like Gladwell makes it out to be (saying that "there is no way to know who will succeed at it and who won’t" and that "college performance doesn’t tell us anything").
The rhetoric in the article is a little ridiculous, but for a moment let's gloss over his glossed over details:
Professional football teams have an extreme financial incentive to accurately predict an athlete's potential. They have a sizable data set to examine, and a lot of money to spend, but there have still been some very notable failures.
Our education system does not have the resources of the NFL to determine who will be a good teacher. But even if they did, it would be a misappropriation of funds. Instead of focusing on pre-facto credentials (graduate degrees, etc.), we should put them in the mix and see how they perform, and pay the best teachers accordingly.
These seem like the most salient points from the article, and I don't think any of them are particularly false or oversimplified. If you know a lot about something and the article is for a general audience, it's bound to seem like a bunch of outright lies. Life is never as elegant as the New Yorker makes it seem, but that doesn't mean it's not entertaining (and sometimes informative) to read.
"Our education system does not have the resources of the NFL to determine who will be a good teacher. But even if they did, it would be a misappropriation of funds. Instead of focusing on pre-facto credentials (graduate degrees, etc.), we should put them in the mix and see how they perform, and pay the best teachers accordingly."
What's also interesting is how having the top pick cripples the drafting team due to the ridiculous salaries given to unproven talent. If I had the top pick, I'd trade it every time. 50/50 chance isn't good enough for allocating 8 figure salaries. The NFL needs a rookie scale like basketball.
“Mind you, that’s not great feedback...The perfect way to handle that moment would have been for the teacher to pause and pull out Venisha’s name card, point to the letter “V,” show her how different it is from “C,” and make the class sound out both letters."
Unless, of course, all the other kids start calling out their own names, then other words, and she does lose control of the class.
It's an excellent article. But the problem I've always had with the kind of analyses portrayed by the UVA team is that it's too easy to sit there and sound smart by making these kinds of criticisms. Similar criticisms could be made about the math teacher, whose style is more like mine was in the community college physics classes I taught, but I'll spare you, dear readers of this comment.
I think that it's better to think of teaching like trying to find the best-fit line through a cloud of scattered points. You can stand there and say, "the shortest distance to the next point is actually over here..." but completely miss the idea that if you are trying to maximize a long-term result, you may not want to draw your line in that particular direction.
I remember being asked to "criticize" papers in college. These papers were always classics in the field, in many cases standing up for decades. It always made me uncomfortable, though I eventually settled into a rhythm of simply automatically and mindlessly "criticizing" that "they should have done more". Dress that up in three paragraphs and you're done.
And this is the same thing. Criticizing that somebody should have done more is a null criticism. What needs to happen in any environment where you have limited resources (limited pages the journal will publish, limited time with the children) is that if you're going to say somebody needed to do "more", you need to also criticize what should be removed to make room for this "more".
By that standard, I doubt this teacher could have done much better. There's a balance to reacting to the kids, but you also need to get through the curriculum, which ultimately is there for a reason. (Most of us would probably agree there's too much emphasis on curriculum today, but the solution is not to completely ignore it.) Reacting endlessly to the students comes with its own problems.
I should have been nailed for my formulaic criticisms. But then... students "criticizing" the top 0.01% of papers shouldn't be expected to produce useful results anyhow, any more than this criticism of the teacher does. There's only so many pages and so much time.
I must admit the part about those people criticizing the videos really doesn't sit well with me. Do they assume there is one perfect way to teach? Are they working towards establishing the "fascist" school of perfect teaching, that all future teachers will have to rigidly adhere to?
Maybe pulling out the letter V would have distracted too much from the letter C, or whatever. I am afraid such people are just parasites to society, somehow they managed to tap into tax payer money and get paid for just sitting on their ass all day talking bullshit. Probably standing in front of a class directly they would be miserable failures. Sorry for sounding so harsh, but my spouse is a teacher, so I guess I am a little too emotionally involved...
They're trying to systematically determine what works and what doesn't. They've got to review a lot of different teachers in the same and different situations. A teacher who wants to do better would greatly benefit from specific, actionable recommendations. Creating tools that can be used for deliberate practice is useful work, especially if the results can be widely distributed.
While I'm not a fan of Gladwell's analysis (I agree that he has a tendency to pick data that supports his point, while ignoring evidence that contradicts it), I did like the message of this piece.
I.e., we should let more people try their hand at teaching, with the idea that most will probably not be good at it.
This assumes, of course, that none of the failures are allowed to be tenured, and that teaching can be made an attractive career choice.
What about variability and stride? A teacher's performance is likely to vary daily, as well between years. How do we know, rejecting a mediocre teacher, that she wouldn't have blossomed into a superstar? How do we know that the trigonometry teacher won't become cynical with age?
The second problem can be solved by gathering data every few years, but what about the first?
A lot of people already have. The fact that school voucher systems and charter schools exist in some areas is testament to their heroic effort. However, it is too little and maybe too late.
Definitely too late for kids already in school, but I would wager that humans will be educating their kids in schools for hundreds more years (maybe more). This would be like Edison giving up on the light bulb because it was too late for the kids who had to go to bed at sundown.
To the other commenters, this article isn't about college football. Most of the substance here is really obvious to anyone with a modicum of interest in teaching, but unfortunately that is what happens when you deal with politics and bureaucracy. You have to state the obvious, again and again and again.
Teaching is one of my secret passions. I have always been able to make people understand difficult things by talking to them and watching how they respond. When I study, people often ask me for help instead of the teacher/lecturer. I love it.
The thing is, being a teacher would be an incredibly stupid profession for people like me. I will earn twice as much money doing something else, navigating a tenth of the bureaucracy and being able to call the shots without bothering with twenty sorts of political correctness. There is no room in a normal Western school for my style of teaching. I'd be surrounded by imbeciles (imbeciles with authority) and not get paid for my effort.
There is something deeply wrong with our system when it deters people who actually want to do good. There is definitely a business opportunity here if I'd want to start teaching privately or start some private school. Rich people see what's wrong here, and are probably willing to pay to fix it.
Here's what needs to be done: Pay more to the best teachers (at least twice as much as they get today), abolish cartels and unions among current teachers, measure results, remove bureaucratic barriers to alternative styles and techniques, and drop all requirements of political correctness.
Most of these requirements are, for different political reasons, impossible to achieve in a publicly funded system. But if some private school starts to produce much better students than the public system, there will be chaos.
> But if some private school starts to produce much better students than the public system, there will be chaos.
I'm a miltary brat, so I've been in and out of a half-dozen public and private schools in my K-12 education. Everywhere I went there were private schools significantly better than the public ones.
The chaos seems to be confined along socioeconomic lines. The best teachers in the world can't teach kids that don't show up for class, and I'd say America is pretty split as to whether showing up for class is strictly necessary for future success.
I went to a private (school equivalent to) high-school myself..in Norway, and the same holds here. It was about three times better than any public school I'd been to thus far.
Wow, that is one of the worst articles I have ever read by Gladwell.
I generally think Gladwell is overrated, but at least provides interesting anecdotes. I was sorely tempted to quit reading it about 2/3 through, but then I thought "maybe he'll tie together the 'football quarterback selection' story with the 'teacher selection' story in a meaningful way." But no. This would have been more interesting as a piece about either just college football quarterback selection or just the question of incentivising a high level of primary/secondary school education. In the context of the article, they're really not the same question
I'm not against open ended questions and unresolved endings, but the analogy was strained, lacked interesting content. And I am interested in both college football and the "quality of US secondary school education" debate...
The Gladwell love-fest is getting out of hand. I've enjoyed his writing to date for the most part, despite thinking that it was not quite as groundbreaking as some people seem to insinuate. But now, anyone will publish him just because he's Gladwell.
It wasn't really about teaching or football. It was about how being unable to measure performance causes a poor allocation of human resources. Top programmers not getting paid for the value they produce is the exact same problem.
I'm sure his points are interesting, but I'm inclined not to bother to read an article concerning a researcher who hasn't taken the time to learn to speak English. "withitness" indeed. What about "awareness"?
Does it bother anyone else that a researcher has to use "withitness" when other words or definition would have sounded better and read less like pop-psychology noise?
I am a high school teacher and I whole heartedly agree with the sentiment expressed in this article. I don't know anything about NFL so I don't know if the comparisons are accurate, however the observations about good teaching are spot on.
There is no denying that those studying under a good teacher learn more than those studying under a poor/average teacher. There has been a lot said about "withitness" in the literature over the years and it is a quality that one either has or doesn't have.
There are lots of conclusions you can draw from observing teachers and it's quite awe inspiring to see a truly excellent teacher do their job.
@Tichy: Nothing in this article had anything to do with 'establishing the "fascist" school of perfect teaching, that all future teachers will have to rigidly adhere to'. The good teacher qualities that were mentioned included questioning, providing direct responses to student questions and treating learning as a team effort (teacher and student learning together). These are qualities that every good teacher exhibits and all poor teachers struggle with.
Interesting article but I wish the author would be a little more concise. Rambling on about football picks when the topic was teaching was a bit annoying even though he tried to tie the two together.
The topic was inability to measure and rate people for a given profession. The recent (and ongoing) discussion about why great programmers don't get paid more is equally relevant. It's about information asymmetry which prevents efficient allocation of resources. But my sentence there was nowhere near as entertaining as Gladwell's anecdotal writing.
2000: Chad Pennington, this year starting qb for the Jets, doing well.
2001: Michael Vick, who was one of the top few QBs in the league. Of course talent scouts can't predict animal cruelty.
2002: David Carr: Playing backup to Eli Manning. Started for 6 years until an injury sidelined him. Not a McNabb, but far from a Tim Couch
2003: Carson Palmer: 2 pro bowls with the Bungles.
2004: Eli Manning: Superbowl MVP
(here's where players are still too new to tell much about)
2005: Alex Smith: that one hasn't worked out yet and may never.
2006: Vince Young: Promising backup on the Titans.
2007: JaMarcus Russell: Starting for the Raiders
2008: Starting for the Falcons.
Either way, in any non-Gladwellian universe, I'd say college QB scouts are doing pretty well.