Billy Beane's shit doesn't work in the playoffs, the book/movie ignored the Big Three, and information advantages go away.
The startup corollaries: small-bore inefficiencies can't overcome luck and strong competition, don't attribute success to A when it's really the obvious B (i.e. what differentiates your product doesn't mean it's actually better - Gowalla and foursquare is my favorite example here), and people will steal your landing pages because they work better.
What is this supposed to mean? The season gives you a 162-game sample. The playoffs are a coin-flip, by comparison. It's just too small of a sample to draw any conclusions, which is why both old- and new-school baseball guys have an endless array of cliches about how anybody who makes it in to the playoffs has a chance to win.
In short, the A's roster was actually constructed to succeed over 162 games but the compromises he had to make on defense and power pitching to get extra value are particularly fraught in the playoffs, where you're battling the better offensive teams in the league. Of course, the playoffs are a crapshoot (the Cardinals had no business being in, let alone winning in 2011), but the compromises Beane had a direct effect on their heartbreaking playoff losses in the early 2000s.
These are not inherently better offensive teams. That's the whole point. The 2002 A's were a good offensive team because they were better at evaluating offense than the rest of the league.
The reason the A's didn't win the World Series is the same reason the Cardinals won the World Series in 2011 and 2006, but not 2004. It's the reason the Giants won it in 2010. The playoffs are a crapshoot. It has nothing to do with whether or not sabermetric analysis is valid for regular season games, but not for playoff games.
This is not true at all, though - what compromises exactly did they make in power pitching? At most, the team structure would decrease their chances in the playoffs by a few percent. Any analysis based on strictly playoff results is going to be rife with small sample size problems...you can come up with pretty much any conclusion you want based on a sample of a few series.
I don't know if you read the book (Moneyball) but the main point that I took away from it was that the Oakland A's roster was made to win games.
The whole point of the book was that winning games was done by getting on base and the people who get on base more were better to have on a team than someone who was good at fielding (which the book said didn't matter).
The reason the playoffs were such a crapshoot wasn't because Oakland had a worse defense or pitchers but because the amount of games being played was so small that the results could go either way. as the teams in the playoffs were generally of similar quality.
I agree. Even a 162-game sample isn't enough to get any great level of confidence, but it is certainly better than a 3-5 game sample of the first round or 4-7 of subsequent rounds.
Beane and the A's were mostly focusing on offense, because that's were they could find value. It's significantly harder to find value in pitchers, and pitchers can help damper some of the "luck" factors in the postseason, though it's very much open to randomness.
Modern baseball management is focused on building your team to win enough regular seasons to make the postseason, and then hope luck falls in your direction. If you can couple that with great starting pitching, you can reduce the amount of luck you need.
I'm not sure how relevant the techniques in Moneyball still are. At this point, every team is looking at Sabermetrics, and some of the high-value stuff isn't there any more. For example, getting on base isn't worth as many runs in 2011 as it was in 2002 because there are less home runs hit.
"Beane and the A's were mostly focusing on offense, because that's were they could find value."
This is not true. One, the big three of Zito, Hudson, and Mulder is one of the best front ends to a pitching staff in baseball history. Moreover, Isringhausen, Bradford, Mecir, and others on that staff were all players undervalued by the traditional ways of evaluating players. They had a team ERA+ 122. That is not a team which is "mostly focussing on offense."
I worded my post poorly. I meant to imply that their big value finds in free agency came through offense. They were able to adequately replace Damon and Giambi (Jason) with much cheaper counterparts. It would have been much more difficult to find cheap replacements for the big three.
Zito, Hudson, and Mulder were all draft picks. I think Moneyball did discuss the A's strategy to draft college pitchers instead of high school, as Beane found high school pitchers to be over-valued.
Actually, the A's did have a streak of playoff appearances 6+ years ago under Beane's system[1].
Reading the book, Moneyball, I came away with the conclusion that the success of Beane's system boiled down the fact that they started measuring everything and making empirical decisions based on data rather than emotion or "gut" checks. In turn, this let them find value where others saw failure. In a competitive environment with limited resources (players Beane's instance) this system will work up to the point where few, if any, try it--meaning lots of overlooked value exists.
One point at the end of the book was that the Red Sox, when they hired Theo Eptstein as their GM, brought in one of Beane's assistants to implement the system the A's were using. As more teams start using it, the quantity of undervalued players decreases because everyone starts valuing the same thing.
Start up takeaway: Measure everything and make decisions based on empirical results instead of "gut" checks. That's really what it boils down to. I'm not just talking about the front-end stuff like landing pages, conversion rates, clicks, etc. Measure it all, response times, database operation times, network transmission time, average request size, whatever you can get your hands on. Having all that data will allow you to make informed decisions and identify places for improvement to increase value.
This doesn't make any sense. Billy Beane's "shit" either works or it doesn't. No one is claiming that good analytics guarantee success in small samples, but if properly evaluating talent improves chances of success in a 162-game sample, it improves the chances of success in a best of 5 or best of 7 series.
Maybe you're arguing that the amount of luck influencing the outcome of the playoffs is much higher than the amount of luck influencing the regular season; this is obvious. What it's not is a good argument for not using all available tools to put the best talent on the field.
Not necessarily. For the season as a whole, you don't really care about individual contests -- you're just looking at beating the averages.
For a playoff, things are different... you're in a match-up against a single team that is better than the average team. I would argue that money and luck plays a much bigger role in the playoffs for a smaller team. If you're playing the Red Sox or Yankees, chances are that all of the key players qualify as "best talent", while your budget forced you to choose your investments more carefully or choose players with less-known ability.
>I would argue that money and luck plays a much bigger role in the playoffs for a smaller team
Luck plays a much bigger role for all teams in the playoffs. That's the nature of the small sample size. I'm not convinced that money plays a bigger role in the playoffs for a given team than in the regular season.
One valid consideration, in the context of the 2002 A's, could be that they built their success by adding wins on the margins. Put another way, the 2002 A's on average were one of the best teams in baseball, but the playoffs put greater emphasis on the "top" of the team, or the best players. That A's team was short on all-star talent, but made up for it by adding wins at the margins. But adding 2 wins by intelligently selecting a fifth starter doesn't matter in the playoffs, because said fifth starter will pitch out of the bullpen, if at all.
However, baseball is radically different than it was in 2002 and I don't think that still applies to the same extent. Almost every team is using the kind of player evaluation that Beane was using in 2002. There's still some fogey Joe Morgan types that just happen to be running baseball teams now, but they are by far in the minority. So now you have to be even smarter than Beane was at the time (see Friedman and the Rays) to be competitive on a budget. But the way Friedman et al. are doing it now isn't subject to the same implications as the way Beane did it when RBI was considered a good metric for talent evaluation.
The startup corollaries: small-bore inefficiencies can't overcome luck and strong competition, don't attribute success to A when it's really the obvious B (i.e. what differentiates your product doesn't mean it's actually better - Gowalla and foursquare is my favorite example here), and people will steal your landing pages because they work better.