Hacker News new | past | comments | ask | show | jobs | submit login
Inside the Secret World of the Data Crunchers Who Helped Obama Win (time.com)
285 points by sek on Nov 7, 2012 | hide | past | favorite | 133 comments



I think polling on such a detailed level is fundamentally changing the democratic process. In the past, politicians had to be leaders - present a vision that was quite possibly unpopular to start with, and persuade people to come around to it. (See e.g. JFK and the space program.) Nowadays they're becoming expert hagglers: keep this mini-group barely content, say exactly the right nice things to the other, ignore a third altogether because they're 10% over in enemy territory. I don't know where this is leading to, but it's definitely a big change.


I disagree, politics has always been about haggling, you cannot do politics if you don't know how to build a coalition. A politician cannot represent 100% of the people that is why there are political parties and politicians of different ideologies. I think if you look a little deeper all the JFKs, Reagans and Clintons were hagglers and negotiators first. A good politician is an expert negotiator and the very best are exceptional leaders. Its been the same since the first democracy was born. The only change now is that it is very easy to identify who will part of your coalition.


The vast majority of the population are influenced by some sort of benefits the government "offers" whether they be tax breaks, healthcare, welfare, gun rights, subsidies, higher pay for teachers, etc...

Because of this I think that this sort of micro-haggling can happen. Instead of offering a broad plan that is ideologically consistent, and then trying to convince the population to go along with that - politicians can turn the knobs of the different constituencies by holding out the carrot that matters to that group of people.


If your contention about voter bribery is true, how come red states are the ones taking the biggest government benefits relative to their taxes and blue states the opposite?

I think voters vote for ideals rather than as a business proposition, and there are polls that back that up. The idea that 47% of the electorate is 'bought' was pretty thoroughly slammed in the media the last couple months.


I'm not really taking it to the level of bribery. I just think that by analyzing the different micro-constituencies and what makes them most concerned - social security, taxes on high pay, auto industry jobs - you can better tailor your message to them.

While I agree that voters vote more based on ideals - I know I do - if there are enough 'business propositions' that go against you, that is most likely going to be an affront to your ideals. If your job relies on corn subsidies, you more than likely agree with the idea of crop subsides, and therefore will vote for the candidate that supports your "ideals".


Sure, but even if we slice it by constituencies, medicare recipients overwhelmingly voted for romney-ryan while the 18-44 set voted for Obama. I still think ideals come in way ahead of "what's best for me personally".


Red states / districts have significantly lower population per area, education levels, and incomes etc. However, the republican party is vary heavily influenced by it's donors which tend to be either retired or affluent. From this perspective you can see why so many of there policy's are setup to hurt the vast majority of their voters.


The problem with that is that the media is not that stupid. If you don’t have a consistent message you will be called out on it.

There are forces counteracting that, you know.


Ronald Reagan's electoral graph would like to disagree.


Winning 49 states is VERY much not equivalent to winning 98% of the popular vote.


Indeed. Even Mondale got 40% of the vote. The idea of Reagan as a qualitatively different "uniter" and "great communicator" is mostly a myth. The truth of the matter is that 1984 represented the apex of the "southern strategy", the republicans having shorn up the southern white rural vote pioneered by Nixon, but not having lost the middle class of the northeast or west, nor yet dealing with the emergent minority demographics of the modern world.

And he was a great candidate facing a pretty mediocre one as an incumbent in a growing economy. It was a perfect storm of circumstance, not a unique moment.


The oposite opinion can also be argued, that now politicians are a lot more connected to the populace, and that the will of the people is now represented much better than it was in the past.


How does a politician become "more connected" to the populace if said populace is only encountered as an integral function of a wider statistically significant effect?

"Dear Candidate, you will have to see people in that part of the country, not because you care, but because our data has shown that a similar visit before was statistically significant enough to matter."

On top of that, this data-driven approach actively undermines any connection by reducing it to a functional one. There isn't even the pretence anymore that the candidate cares for the given group due to intrinsic values or some sort of connection other than statistics.


> "Dear Candidate, you will have to see people in that part of the country, not because you care, but because our data has shown that a similar visit before was statistically significant enough to matter."

When has a candidate ever campaigned in a place for any other reason other than an attempt to snag votes? The advent of computer-driven data didn't suddenly steal the innocence out of campaigning.


Let's take it to the logical extreme. Say the candidate did exactly what the statistics said to do on every issue and that they had no vision or leadership whatsoever. That candidate would be the closest a representative could get to the will of the people. It would also be the closest a representative democracy could get to being a direct democracy.

While I can appreciate there is value to having figureheads around, it's hard for me to look at a system approaching direct democracy and feel that it is worse.


Since when is direct democracy completely void of any form of value? Following your "extreme", then, means that every election means simply projecting all your private hopes onto someone completely void of any coherency, other than that projected onto them.

Statistics reflect groups already present or in the process of development, which formed due to previous work (be it a former presidency or the opposition thereof). Giving statistics free reign does not approach direct democracy, because it merely manifests what was already there, or -- if you want to be more dystopian -- they will create through their own precriptions what they define (→ theory effect).

The whole point of holding elections is to find a compromise that works for the majority -- if not most -- of the people, and for that you need a program that then creates groups. If you however campaign solely based on the already existant groups, you will cement the already existing groups (and schisms) without the chance to actually initialise any new group formation processes.

I don't understand direct democracy as a system where the candidate serves merely as an empty canvas on which every individual voter (or group of voters) can project their preconceived opinions without even having to engage in any form of idea-led political discourse.


Under direct democracy, there wouldn't be a candidate at all--the citizens propose and vote on every issue themselves, instead of electing a representative to do it for them. It's been done before, and the principle argument against it is that it doesn't scale (thus representative democracy).

"Merely manifesting what was already there" is exactly the point of democracy: the majority rules as opposed to some superior-minded entity. Your argument seems to be that representative democracy is better than direct democracy because compromise wouldn't exist without representatives or people wouldn't factionize without them. I find those claims dubious, but we're all entitled to our opinion.

What I think lies beneath your argument is a form of the old classical AI versus statistic AI debate. The classical side asserts that there's not much scientific value in statistical techniques because they don't increase our understanding. The statistical side emphasizes pragmatics and has the best results. Interestingly, even if your side were right, it's going to be (and is already, evidently) steamrolled by the other side simply because they have the results. It doesn't seem impossible to me that there could be something valuable about sharing values with candidates, but the candidate that employed the statistical method better won. As long as that remains the case--in other words, until voters vote based on actual values rather than statistically determined hot-button issues--it's a moot point.


It seems to me that the whole point of having elected leaders instead of making every choice a referendum is to modulate the will of the people, instead of following it blindly.


Correction: the will of those who are financially able to represent themselves in front of politicians. Presidents and leaders don't have time to understand their populace on their own accord these days.


Whether you like or dislike Obama, the single biggest difference between him and Romney is leadership the way you described it -- willingness to stand behind something unpopular.

Winning requires all kinds. Sketch out the broad vision, and then do the little data-driven things that can give you a bunch of tiny little edges on the margin. But if it's not close to begin with, no amount of small edges are gonna help you.

A good analogy would be having good architecture and good code quality in a project -- you need both, but no amount of code quality can save a bad architecture.


> Whether you like or dislike Obama, the single biggest difference between him and Romney is leadership the way you described it -- willingness to stand behind something unpopular.

That doesn't sound completely right to me. Romney's stint at Bain was expected to cost him the presidency (and maybe did) because of the unpopular things he did while there: Fired people and outsourced jobs while making a couple hundred million for himself in the process. Of the many things he's flop-flopped and waffled on, that record isn't one of them. If running for president in a down economy with that around your neck isn't standing behind something unpopular, what is?


Excuse me, political leadership. Romney has, to my knowledge, not once stood on the losing side of an issue out of principle, and if the electorate flips, he does too. Obama has gone out on a limb for things that aren't popular multiple times.

Not being able to change his past at Bain, not that he didn't try, isn't a sign of backbone, it's just how 'the past' works.


> Romney has, to my knowledge, not once stood on the losing side of an issue out of principle

You clearly have an issue in mind when Obama did this?

> Not being able to change his past at Bain, not that he didn't try, isn't a sign of backbone, it's just how 'the past' works.

By running with this past (it's very much an option not to run), he implicitly ran on a non-flop-floppable platform of "I'm willing to do stuff that's unpopular and will hurt".


The healthcare bill is a pretty high profile example of Obama standing by something that's a short-term political loser. Stimulus, arguably, as well.


There's a lot of stuff. In addition to those, these are other examples off the top of my head:

* Coming out in support of gay marriage, which has always lost in state ballots.

* Going after Osama bin Laden, on a 50/50 chance, inside a foreign nation who would have every right to shoot down our invading helicopters.

* The auto bailout, which was not popular at the time. It's amazing how easily people forget stuff like this.


Osama bin Laden is a great example, because it could have gone horribly wrong and made Obama look terrible. The other two are really lousy examples, for two reasons:

1. Gay marriage and the auto bailout are or have been unpopular, but not among Obama's core voting groups. Yes, standing behind those positions will piss off lots of people, but not people who were going to vote for him anyway. The ACA falls under this category as well, in that it wasn't terribly popular, but many on the left didn't like it because it didn't go far enough. No way were they going to vote for Romney because Obama hadn't gone far enough with the ACA.

2. Obama didn't come out in favor of gay marriage until Biden forced his hand, and there's no way he would have. But Biden forced his hand and it turned out well for him in the end.

Obama is just as calculating and concerned about keeping his base happy as any other national politician.


To say the ACA "wasn't terribly popular" is a huge understatement.

The amount of $$$ spent against "Obamacare" before and after its passing easily numbers in the hundreds of millions of dollars. "Obamacare" is/was considered a defamatory label. The president's Chieff of Staff essentially begged the president not to go through with it. 0.00000000000000000 Republicans ever went on board with it. The whole law wasn't even to go into effect until 2014. Presidents for how many decades have tried and failed to deliver health reform?

There was absolutely no short-term (or even near long-term) political gain to be found in pursuing the law. And if the law didn't pass, the president would be in a much weaker political position.

Even by passing it, the president knew the process would kill any remaining political capital he had left and also hand the House of Representatives to Republicans. That much was not rocket science.

What is amazing is that he chose to pursue it anyway. It's even more amazing that the law passed given all the political and entrenched forces against it.

So, to be clear, the ACA was emphatically not popular in any way. I still remember all the anti-Obamacare ads that showed up on Youtube videos. The law was villified to kingdom come and it would be a tragedy if that fact is forgetten.


Don't confuse volume of attack with number of attackers.


The president bet all of his political capital, the House of Representatives, and potentially his entire presidential career on passing 'Obamacare'. I don't know of a better example of acting in spite of the political ramifications.


I'll agree that he dragged his feet for the right moment on gay marriage but the auto bailout could have SO easily gone the wrong way and Obama would be married to it. As in, he's catching the blame for decades of business decisions, in exchange for trying to help. But he did.


Are we sure the auto bailout hasn't gone the wrong way? The problem with bailouts are two-fold:

1. They protect jobs, but often at a much higher cost than is justified. Admittedly, I don't know the numbers here, but I recall something about the ban on foreign tires protecting US jobs at a cost of $900,000 per job saved.

2. They introduce moral hazard. See our repeated pattern of bank bailouts for examples. If the auto bailout did cause this problem, we won't know about it until much later.

A defining feature of capitalism is competition, which means economic winners and losers. If we protect companies from the need to restructure, even for a noble purpose, we may cause more problems down the road. The auto industry wouldn't have ceased to exist if two companies had gone through bankruptcy. Other automakers and manufacturers would have bought those assets and run them more efficiently. And those other companies are exactly the type you want to be doing this, since they didn't need a bailout in the first place, right?


That's a longer term than political success/failure and a different conversation.

I don't think the moral hazard argument holds for the auto industry like it does for banks, and the auto industry is strategically critical to have around and in good health. But there are valid arguments that disagree.


Those are all valid points, and I thank you for the civility :)


Hm, you're right. I was actually under the impression ACA was fairly popular among the relevant groups, but it wasn't. http://www.washingtonpost.com/blogs/the-fix/post/six-charts-...


This is called 'science'. They used these techniques back in the days of JFK, too.

The difference is that now, campaigns have access to unprecedented amounts of audience data. But make no mistake, it was all dog and pony back then, too.


> They used these techniques back in the days of JFK, too.

No, the contemporary approach is based on a series of tools, such as focus groups, which are a very new phenomenon in politics. The first national campaign where these figured prominently was 1992, I believe. (With e.g. "soccer moms" among the archetypal mini-groups.) Their sophistication increased by leaps and bounds up to the level described in this article. And they are quite game-changing for the nature of politics.


That's how politics has always worked at the city scale and lower. Now it's happening on the national scale, using polling and computers to gather and analyze the data, instead of using party members to gather data face-to-face and a party organization to collate and understand it.


This may be myopic or off-base, but I feel like JFK had two big advantages with the space program that helped make it happen:

1. His constituents were terrified of the Soviet space program dropping something nasty on their heads, and didn't want to be behind the curve in that arena. 2. His assassination effectively canonized the program. (I think he'd argue my terming that an "advantage.") Cutting it after his death would have been political suicide. Where space exploration often has the same inherent challenge as nuclear power–specifically that the length of any serious endeavor will span several presidents, congresses, senates, to say nothing of economic cycles, (see CxP)–the Apollo program, despite being 2.2% of all federal outlays, had broad support.

There were numerous other factors, but I think those two played a big part in its success.


Just like any new technology when it's introduced, the value of this micro-targeting strategy will be overstated at first. But over time people will realise human nature has stayed the same, and this technology will find its place as just another tool in a politician's utility belt.

Perhaps we've already seen it misused on Romney's part: depending on where in the country he was speaking (say, southwest vs. rustbelt) he would take a different tone on the topic of immigration. In the age of instant news, flipflopping like that in the name of "targeting" your audience can have major consequences.


Well, it seems like a big change. In the past, your observation (about haggling) was applicable to local elections; perhaps now it simply escalated to a larger, country-wide, scale?


Agreed; zeteo is looking at the past thorugh some decidedly colored glasses. That's never been the way it worked. Elections have always been won via "haggling", whether that happened in the proverbial smoke-filled rooms of party conventions. Remember that the expectation of a general election being "close" is a fairly recent one. From 1865-1932, for example, the president was effectively picked by the republican convention delegates.

Winning an election through careful attention to the polls simply means you're tailoring your messaging and policies to the opinions of the demographcs that matter most to victor. How is that not "democratic?".

If your complaint is that some demographics matter more than others, then address your criticism to the electoral college, not the strategy picked by the winning candidate.


I think you underestimate the sophistication of techniques of the past. They may have had less raw data, but what makes you think they didn't know precisely who they were talking to?


I find this really interesting, but I'm unclear on the proof that this made a big difference for Obama's campaign success. I'm sure Romney had people crunching the numbers as well, and he actually raised more money than Obama.

This was a pretty narrow victory, there are a lot of structural, ideological, and demographic biases at work here, and Obama did worse among most groups than he did in 2008.

Put another way, is there any evidence that Obama would have lost or even done much worse without this data crunching effort?


There's at least some evidence that, in fact, the Romney campaign fooled itself into believing the "unskewed" polling models which simply (and incorrectly) disbelieved the demographic makeup of the polls. So yeah, it's hard to believe the republican campaign could have run a heavily data-driven campaign.

And the evidence that Obama would have actually lost is obviously not going to turn up. But you can look at the relative performance in "swing" states vs. the general electorate for hints. He clearly outperformed among the demographics that ended up deciding the election, as witnessed by the very large electoral victory on the basis of a very narrow (currently <2% I believe) popular vote margin.


There's at least some evidence that, in fact, the Romney campaign fooled itself into believing the "unskewed" polling models which simply (and incorrectly) disbelieved the demographic makeup of the polls. So yeah, it's hard to believe the republican campaign could have run a heavily data-driven campaign.

Sorry, what is that evidence? The fact that they were saying they thought they would win? The fact that they didn't give up? Did you really expect them to? It's important not to confuse the public behavior of a campaign with their internal beliefs. And Romney really could have won this, had things gone a little differently.


The late attempts to to throw campaigning and advertising into states late Pennsylvania and Wisconsin (which, by the now-known-to-be-correct polls, were basically unwinnable) are probably the best evidence. There's also the more squishy testimony of journalists and pundits, who reported on genuine surprise coming from the senior Romney people about the exit polls. Karl Rove's freakout on FOX last night probably also counts.

None of that proves anything. But I think the burden of proof really goes the other way: the Obama campaign clearly seems to have played the moneyball correctly here. If Romney's folks knew they were going to lose, you'd have to show that.

And I don't know how to interpret "Romney really could have won this, had things gone a little differently". If you're talking about polling could have been different had he been winning, then of course: we'd be having the same argument in reverse. Though again, the evidence seems to say that Obama would still have outperformed the popular vote margin and thus have still been running a better data-driven campaign.

If you're saying that the data suggested by the end-campaign polls could have supported a Romney win, then no: that's just wrong, as evidenced by the discussion we're having. That kind of thinking is exactly the kind of "faith-based" notion that the Romney campaign is accused of relying on. The data showed a clear Obama victory. The data was right. Period.


Actually...

The Romney campaign is reported to have believed they were winning late into the day yesterday.

Based on what their model was telling them.

Not "faith", but data. A model. Math.

I have a strong suspicion in there's a good book in this election- how one set of Big Data led one team in the right direction, and another set of Big Data led the other team in the wrong direction.

This to me is the absolutely crucial thing to keep in mind. You can have your model and have your data, but that doesn't mean you've picked the right model and the right data.

Not the first time that lesson has been demonstrated, and it won't be the last.


The big difference in the models was their assumptions on voter turnout for each side. You can precisely measure what people prefer, but you can't precisely measure whether they will show up next week. So it was the 'art' part of modeling, not the 'science' of statistics, that the conservative pollsters blew.

It's hard to know, but one reason for the strong turnout might have been better get-out-the-vote efforts of the kind described here.


Follow-up articles seem to be indicating the Romney team was indeed wrong about turnout in each direction. Dems turned out more than they expected, Republicans less.

Not unreasonably. Most public polls supported that belief. http://www.washingtonpost.com/blogs/election-2012/wp/2012/10...


No, that's just wrong. Most "public polls" (rather: the median pollster -- obviously there were outliers in both directions) supported an Obama victory and a demographic and partisan makeup of the electorate that matched closely what was observed in the election. To claim otherwise is just silly, and perpetuating the "unskewing" nonsense that was proven incorrect.

If you're saying that the Romney campaign had "reasonable" justifications for disbelieving the data they saw, then I agree. If you're claiming that they actually had data the supported that position, then you're wrong.


We're talking turnout now, not preference. No one is talking unskewing.

The polls did in fact show that Republicans had higher enthusiasm, which is valid evidence for expecting better turnout. The better turnout did not materialize, but it was completely reasonable to believe that it might.


No, no, no. Polls include a prediction of turnout. That's the point of their likely voter models. They were telling you who was going to go to the polls already, yet the Romney campaign (and essentially all of the conservative establishment) chose to willfully ignore that data on the back of some cherry picked numbers (e.g. Rassmussen) and a set of "unskewed" models that were simply wrong.

There was never any data for that. It was all (incorrect) analysis. So don't say that there was data to support the position, there wasn't.

And the "enthusiasm" numbers were from early in the spring, before likely voter probing can be done. That stuff all disappears once the LV polls start coming out post-primaries.


No, it doesn't. Some polls ask about enthusiasm right up until election day.

Again, this is not about unskewing. This is about data as it was actually reported directly on the page.

Here's one of the polls directly referenced in the link I included in previous post. NPR Poll Oct 23-25, 2012. Just before the election. http://media.npr.org/documents/2012/oct/NPROctpoll.pdf

There's a 10 point scale that gives a 10 percent difference in enthusiasm to Republicans at the "10" level.

Of course that advantage vanishes if you include "8-10", which could be the sort of mistake Romney's internal pollsters made.

The poll only includes Likely Voters.

So there was data in public polls just before the election reflecting an enthusiasm advantage for Republicans. To say there wasn't is simply wrong.


I'm just at a loss. The poll told you how many voters it thought would appear, based on factors including internal modelling and the enthusiasm quesiton. And your argument is that you should throw the aggregate data out based on that one number, that they already included? That's insane; it's quite literally cherry picking.

And, of course, it's just flat wrong. That you would defend this behavior is just beyond me. It was a mistake, it wasn't ever a reasonable interpretation of the data, and plenty of smart people said so at the time.


The enthusiasm is a measure among the reported data. Not among the survey sample. The data was culled for likely voters, then the enthusiasm spread was among those that survived the cut. At the bottom of each page it says the results are all from the weighted sample.

The concept here is that "likely" is a continuum, and the Republicans had an advantage at the extreme upper end of it.

EDIT- Adding a note and then moving on from this. I understand you're saying the enthusiasm is baked into the final preference percentages, but that's not the case.


I was interpreting your stance as a verdict on the Romney campaign's behavior over the last couple months, rather than just last night. And a month or two ago, they were behind according to the polls but they definitely could have still won.

Pennsylvania and Wisconsin I think are evidence of exactly the opposite: they knew they were going to lose Ohio and they had money to burn, so they threw a hail mary. Because why not?


The Romney campaign’s equivalent was apparently some system called “Orca”, which involved volunteers on the ground sending information to a central location about the conditions that they saw. It wasn’t as extensive, and now we know that it gave misleading information to the campaign.

Two other things to keep in mind:

(1) The Republican leadership has been really really invested in the “fake it ’till you make it” school of persuasion: if you act like a winner, this will persuade the people who want to get behind a winner, and then you will win.

(2) Democratic operatives have known for decades that the demographic groups more likely to vote for them are also the groups that are less likely to vote at all. The Clinton-era solution to this problem was to pitch a message that was more likely to resonate with the population that was already voting; the Obama-era solution is to get more of those unlikely voters into the voting booths. Republicans don’t have as much headroom to improve their turnout.


It may have been a narrow victory in the popular vote, but it was an absolute stomp in the electoral college. Obviously we can't go back in time and try it a different way so you're never going to have any "proof", but there was clearly a strategic advantage by the Obama camp in the battleground states.


"I'm sure Romney had people crunching the numbers as well, and he actually raised more money than Obama."

Actually if you look at direct contributions, Obama raised nearly double what Romney brought in: http://www.opensecrets.org/pres12/index.php

SuperPACs allowed Romney to compete financially, but I don't think the number crunchers they talked about in the Time article were involved in the PACs


There is no way to disaggregate all the hundreds of factors that went into the 2012 election. So we'll probably never be able to say "X% of Obama's victory came from data mining and Y% came from hiring an addition Z Field Organizers".

But given the disadvantages Obama had going in, he definitely performed towards the top-end of what could be expected.


There is substantial and consistent reporting that Tuesday night's results were a surprise/shock to many Republican candidates and the associated outside groups. We know that they did a lot of number crunching themselves, so the implication is that the quality of Obama's data operation was superior--in that their predictions more closely resembled the actual results.


I think you assume that all "number crunching" is equal.


I absolutely do not, but how do you know which campaign had better number crunchers? You could go by who won, but you don't have a control; how do you know who would have won without the supposed advantage of the supposedly superior number crunchers behind the winning campaign?


I heard that Romney's number crunchers believed that they were winning till late evening Tuesday.

What other blunders did they make early on in the campaign?


>>This was a pretty narrow victory

What exactly is "pretty narrow" about a 303-206 electoral split?


I think he meant population-wise which doesn't reflect in the electoral count except in analyzing effects of 3rd parties on results.


The gist of this article is that Obama was able to use a combination of data mining and creative marketing to pinpoint impressionable voters in crucial areas with a view toward manipulating the electoral college system to his advantage. This is yet another reason to toss the electoral system in the garbage and set it on fire. In no event should a relatively small subset of the population in a handful of states be responsible for choosing the President of the United States.


Had we used a popular voting system rather than electoral college system, this system would have been expanded to every state in the nation, and Obama would likely have won by a far wider margin than he did.

Again, this system was about identifying impressionable voters and marketing to them properly. The location of swing states is simple and public information, not proprietary.


"Had we used a popular voting system rather than electoral college system....Obama would likely have won by a far wider margin than he did."

That is quite debatable. There is no way of knowing how many people simply don't show up to the polls because they live in a state that leans heavily toward the opposition and therefore know their vote will have exactly zero effect on the election.


The problem with a straight popular vote is that it is much more susceptible to attack.

Take a jurisdiction like Chicago which is known for voter fraud and high turnout among the dead. Under our current system, no matter how compromised that system is, its impact is limited to that state.. it can only swing some Electoral Votes (and everything state and local).

With a straight popular vote, that compromised system would put the overall system at much bigger risk.

The Electoral College serves as a corruption firewall between systems.. not perfect, but significantly better than the alternative.


In your Chicago example, the electoral college could easily magnify it.

Imagine, for a moment, that if the kind of voter fraud that Chicago earned a reputation for in the 1960s were to happen in a city like Miami nowadays. So let's say 5,000 fraudulent votes went to candidate A. In a popular vote system, that's unlikely to swing the verdict because that's such a tiny margin compared to the country's entire electorate. 5,000 is less than 0.004% of the number of people who voted in 2008.

But Florida historically polls close - easily close enough for 5,000 votes to swing which way the state goes. And this winner wouldn't just pick up that tiny sliver, they would get all 29 of Florida's electoral votes. That's a much bigger chunk of the pie - 5.4%. Worse yet, in the electoral college getting the votes is a zero-sum game. (Not so in the earlier case because the votes shouldn't have existed in the first place - so picking up a fraudulent vote for candidate A doesn't mean that candidate B lost a vote.) Candidate A getting 29 votes means candidate B loses 29 votes, so the real swing is more than 10%.

Meaning that in this hypothetical, the electoral college managed to magnify the influence of what was a relatively small instance of electoral fraud by a factor of more than 2,500%, into something that could easily swing an election.


Electoral fraud in Illinois would swing a maximum of Illinois's 20 electoral votes, or 29 in Florida

With a popular vote, it would be popular to have millions upon millions of fraudulent votes in a single place, say, Delaware, that would allow the opposing candidate to win even if there was supposed to be a landslide victory for the other candidate.

I think "Firewall" is a good analogy of this.


How, exactly, would one manage to organize millions upon millions of fraudulent voter registrations in a single place?

It would be better if your hypothetical example were at least remotely plausible. The entire population of Delaware is less than one million.

Keep in mind that my example of only 5,000 fraudulent registrations in a single place is already over an order of magnitude larger than notable examples from recent history.


Simply pick a state that uses electronic voting machines, and engineer Malware, or alternatively, hack the central repository.

I supposed millions in Delaware is a bad example, but it's certainly possible in a place like California.


"Take a jurisdiction like Chicago which is known for voter fraud and high turnout among the dead. "

An unproven case of a small number (<100) of possibly fraudulent votes in an election more than 60 years ago, and still going on about it as if it happens every election, and orders of magnitude worse.


We can have a better more granular "firewall". Instead of direct popular vote, simply quantize it into congressional districts instead of just the state level.

Who wins the district wins that EV. Who wins the state wins an additional 2 EVs (ie, aligned with senate numbers). Total would equal (currently) 538 EVs.

Some states already do something similar (see Nebraska, Maine).

Of course, this brings up the perennial problem of gerrymandering... which could be resolved by mandating perimeter to area ratios that keep things from looking too crazy.


I think there are lots of factors to consider, but the electoral college limits, to a certain extent, the "ground game" advantage Obama had. This is making an assumption that Obama's ground game was scalable, because if we went on popular vote, Obama would be able to ramp up that ground game all over the country, just as he did in the swing states.

What the electoral college incentivizes is isolated ad spending and campaign stumping in the battleground states. It's clear that this aspect of campaigning would change substantially if the electoral college were eliminated, so strategic advantages would be sought elsewhere and I think the ground game would become a focal point.


Even more important is that the system is in place -- already in production, tested, and battle-hardened -- and it is in the hands of Democrats.

The smart Republicans will take this to heart and turn their attention towards the 'quants and coders,' and the smart quants and coders will be looking to be employed by them.


1. He would have won on the popular vote anyway. People who were excited early in the evening by Romney's apparent lead forgot to figure the west coast #s into their calculations.

2. You should throw your support behind one of the initiatives to link electoral college representation to the popular vote at the state level, because you're not going to get rid of the college itself for the foreseeable future.

3. Mitt Romney had exactly the same opportunities Obama did and knows exactly how the electoral college works. He just didn't use them. For example, the Obama campaign gambled and bought a lot of TV time early on when it was cheaper, so they got more bang for their buck. It's no good whining about one candidate manipulating the electoral college system, as if it had somehow been kept a secret from the other candidate. Romney was able to figure out the GOP primary system, which is at least as complex. Every student of American politics understands how the electoral college functions.


This presents elections as a system to be gamed, rather than an earnest attempt to learn the will of the people.


Elections aren't about learning the wills (plural! very plural!) of the people, it's about making sure power changes hands often enough (peacefully) that systematic suppression of minorities is made very difficult. As for elections, it's the vetting of the candidates that's the important bit.


> In no event should a relatively small subset of the population in a handful of states be responsible for choosing the President of the United States.

I don't know why people love to say this; its simply not true. Just because we can predict which way certain regions will lean before the vote comes in does not imply that those regions don't matter. Does Nate Silver predicting 100% of the outcome of the election mean none of our votes mattered? Of course not.

The purpose of voting is not to "change the outcome election". The purpose is to participate in the process, thereby giving the process legitimacy.


Why would you need a different voter targeting system/strategy for a popular vote? The goal is the same - convince people that would support you anyway but are not probable to go vote to actually get to the polls.

In fact, seems like this system worked just fine anyway - they won the popular vote as well.


Whether or not you would need a different strategy is debatable, but you would almost definitely see one. Voters in less populous states are over-represented in the EC, and urban voters are a key Democratic constituency, so you would definitely see messaging shifting away from things that would appeal to farmers in Iowa, since it would cease to matter (think things like mentions of ethanol as an energy source in stump speeches), and towards things that would drive up Latino turnout in Los Angeles: heavier discussion of immigration reform, for example, which very rarely came up in this cycle -- the foreign policy debate didn't mention drug violence in Mexico at all, this time, I don't think, and instead focused entirely on the middle east.

I think lots of "urban" issues would suddenly be much more relevant to the campaign, particularly on the left; gun control would be back on the table, marijuana legalization would start to be taken seriously as a social justice issue, etc. I'm partly talking out of my ass, but things would almost definitely change.


the electoral college system only constrains the location of targets. without it, you might make the problem harder, but you would not make it impossible.


Not even that constrained. They had a lot of fundraising targeting as well, and that was nationwide.


Obama won the popular vote, so what are you talking about?


http://www.slate.com/articles/news_and_politics/victory_lab/... - Slate article from Jan 2012 "Project Dreamcatcher How cutting-edge text analytics can help the Obama campaign determine voters’ hopes and fears."


Interesting.

I somehow got added to the Obama campaign mailing list, and their recent fundraising emails politely noted that their records did not show me having donated any money (I'm not American and don't live in the US). They also said that their records might not reflect my donation if I had made it through a different channel.

However, this article suggests that all the channels go to the same database. I wonder whether one of the following scenarios is correct:

* The database has eventual consistency (maybe my donation through channel X would take some time to be reflected by the campaign team);

* The message was a ruse to throw off independent and Republican analysts to the sophistication of the database;

* It was CYA in case something had gone wrong.


Just for clarity, political contributions directly to a political campaign are a matter of public record in the US. Thus this piece of knowledge was not the result of new data mining techniques, it's been used for a number of years.


They are only matters of public record if the total donation amount exceeds a limit, I think $200.


CYA is most likely, although #2 is possible. Changing names (due to marriage, for example), moving, losing (and thus getting a new SSN) are all possible, and might screw up the database. A small CYA phrase helps a lot.


A similar profile from Mother Jones pre-election: http://www.motherjones.com/politics/2012/10/harper-reed-obam...


Thanks! This is much better than the article in the OP. It also links to a number of nice other pieces on the topic, including 2 Slate articles:

http://www.slate.com/articles/news_and_politics/victory_lab/...

http://www.slate.com/articles/news_and_politics/victory_lab/...


Does anyone have any info about the tech stack?


I don't know about their data scientists, but I've heard from people who know engineers on the campaign that they're essentially a Python shop. Not sure about the rest of the stack...


A more interesting question, perhaps: what data were they mining?


What this article seems to show is that the election was not about political ideology. It was about getting out the vote. The Obama campaign fine-tuned its message to scientifically increase returns in those targeted groups. The Romney campaign played the 'smoke-filled room' game (for all we can tell).

In none of the presidential debates did either man speak in depth about his governing philosophy (or the Fed for that matter). Their talking points rang loud and clear over and over, and finger-pointing abounded, but neither spoke of his core convictions. The campaign speeches were much of the same. For all of the complaints about Romney's lack of details in his economic plan, the President didn't do any better, simply offering a 'stay the course' message (he learned something from Bush).

Therefore, given this article and the lack of ideology involved in the campaigns, I conclude that the voters were not persuaded by philosophy. They were targeted by the campaign so that the powers that be could keep their power.

Looking back to the electoral college results of 1980 and 1984 shows what a strong, clear ideological message can do. Those familiar with Reagan's speeches will know why.

So for those who say that America has embraced any certain ideology, I say that it has not, since ideology hasn't been on the ballot for a long time. People have embraced a man, and the philosophical debate has reached a new low.


I don't know what campaign you were watching, but I heard a lot about governing philosophy. Abundantly from the Democrats side, but also from the Republicans. Of course, they won't go into minute detail about policies on the national level; most people simply don't want to hear that. But it seems clear to me that Obama's political philosophy resonated with more people than Romney's.


ITT we confuse talking points and identity politics with PHILOSOPHY.

Jesus. You know the intellectual landscape is bleak when even hackers are willing to entertain the notion that democracy/mob rule has anything to do with philosophy, or that politicians operate in line with some sort of rational ethic other than straight forward power broking.


It's so easy to play the cynicism card that everything is pandering to mob rule, but if you listen closely many of Obama's major speeches lay out his governing philosophy. I'm a bit too lazy to dig up exact quotes, but his "you didn't build that" speech is a good example. Also parts of his victory speech just last night, and his DNC acceptance speech. It's actually kind of funny that you say he never lays out his governing philosophy, because one of his biggest criticism was that his speeches are too high level and never delve into specifics. Just goes to show that people will rationalize whatever position they already believe.


That stuff is so trivial that it's hilarious to call it a philosophy. By these standards, sponge bob has a sophisticated philosophy.


>I conclude that the voters were not persuaded by philosophy.

Very astute, Plato.

Welcome to planet Earth. Hope you enjoy your stay.


It's interesting that they mention email lists - I tried unsubscribing several times from the President's list, and it never worked. Anyone else notice this?


Anyone have additional insights to what is meant here?

"...the campaign’s Quick Donate program, which allowed repeat giving online or via text message without having to re-enter credit-card information, gave about four times as much as other donors. So the program was expanded and incentivized."

- How did the Quick Donate program work? Like Amazon's 1-Click checkout?

- What was the call to action for Quick Donating again?

- How was the program expanded/incentivized?


Nate Silver made an interesting point a while back when he compared the political data revolution to the Sabremetrics revolution in baseball.

In baseball, the journalists and pundits were way ahead of the professional operators in moving from gut calls to data-crunching, while in politics, it's the opposite - what Silver and other public number-based forecasters do is commonplace inside the campaigns.


Fascinating stuff... and something I want to learn more about.

Can you please recommend me books/software applications/online courses for doing the work these guys were doing for Obama campaign?

(Just to clarify: I am a developer and I grok SQL. So that part is covered. Other parts... less so.)


I recommend Collective Intelligence, published by O'Reilly.

If you look online there's Ruby source code examples to go with the Python in the book.


I'd sure like to know more about their methods and algorithms. What exactly "is" big data is glossed over so much that I can't tell if they are doing something unique here or simply applying a large degree of automation.


Want a future in campaign management?

Cut your chops with an e-commerce company. While there learn big data, how to test, and read into behavioral economics, and you're set.


"A politician thinks about the next elections — the statesman thinks about the next generations."

There are no leaders anymore.


Too bad the single biggest piece of data here is going to be ignored. If they started competing with Silicon Valley for talent and put thousands of these guys to work in a building somewhere to fix this country, they'd quickly become the 'nuclear codes' of the USA. But they wont bother, they'll immediately forget all of this until the next campaign and go back to politics as usual.


Hilarious!

What gave PA to Obama was the blocking by the courts of the Voter ID act that the legislature passed.

Note that in all the praise about individual contibutors in the article, there is no mention of "bundling" and the routine use of cutouts, both rather unsavory practices that both sides use.

EDIT: this site may prove a useful antidote to the above article: http://www.washingtonpost.com/wp-srv/special/politics/campai...

Romney raised more than Obama in total $$$. Even without the SuperPAC contributions that gave Romney more money, the two campaigns were within about 10% of each other in terms of what they raised.


Obama took PA by five percentage points, which is consistent with his margin in the state-level polls that were taken throughout October. In absolute numbers rather than percentages, his margin is about 300,000 votes. Are you really trying to argue that a voter ID law would have taken enough people out of the electorate to flip the state?


Yes. Fraud in Philly voting is legendary and almost on par with Chicago.

I can't find the registered voter turnout for 2012 yet, but in past years Philly RV turnout has been as high as 108%.


Despite common wisdom to the contrary, documented vote fraud is virtually nonexistent in the United States. Highly (overly) motivated individuals with all the resources of government behind them are unable to find it. You can read a summary in this Justice Department report, http://www.justice.gov/oig/special/s0809a/chapter6.htm , skip on down to "Voter Fraud and Public Corruption Matters".

We do have a fair bit of voter registration fraud, where a fictitious name is registered to vote. This is because registration is handled by the political parties paying contractors per registration, leading to fraud for money. The key is that even if Mickey Mouse, or the roster of the Dallas Cowboys is registered to vote multiple times in the country, they don't actually show up and do it.

There are cases of people deliberately multiple voting, but the most common fraud is a convicted felon who has lost their voting privilege continuing to vote. Some states have strange and complicated laws about which crimes render you ineligible to vote, and many felons are not well versed in the law. It certainly sucks to lose your probation because you tried to do your civic duty.


For highly motivated individuals who are able to find evidence, see this article:

http://www.newyorker.com/reporting/2012/10/29/121029fa_fact_...

Republicans who support tighter voter security say that they are not seeking political advantage. But last summer Pennsylvania’s Republican House Leader, Mike Turzai, was caught on tape boasting to colleagues that the state’s new I.D. law was “going to allow Governor Romney to win the state of Pennsylvania.” Earlier this month, a state judge suspended the controversial law’s implementation until after the 2012 election; a federal court has done the same with South Carolina’s new I.D. law."


Admittedly I only got 2/3 of the way through the article, but I didn't find any cases of voter fraud supported by evidence in that part. The author teases a bit, but they turn out to be clerical errors or similar.

Lots of voter registration errors though. e.g. registrations not culled when the voter moves away.

As far as dead men voting. I'm aware of two in this election. They voted absentee with the reasonable expectation they would no longer be living on election day. Legality apparently varies by state.


I was teasing, too. There is no real evidence of a significant pattern of voter fraud. The outcry is just part of a media campaign by right-wing organizations to justify legal measures intended to suppress voter turnout among Democratic constituencies.


Speaking as a former Chicagoan: the way you swing an election is with the connivance of the people who administer it, e.g., by making sure that the ballot boxes from “unfriendly” precincts end up in the bottom of the Chicago River.

Voter ID does nothing to prevent this sort of thing: the election officials in “friendly” precincts can just wink at forged or missing ID.

And voter ID makes a certain kind of election-skewing even easier, since officials in “unfriendly” precinct can discourage people from voting by rejecting valid ID, or simply by spending so long examining the IDs and arguing with the voters that everyone has to wait for hours in order to vote.


Excellent John Stewart segment on the issue of voting fraud and these voter ID laws. For the most part, not a single example of fraud was found, so it's hard to justify laws that disenfranchise large segments of society.

http://www.huffingtonpost.com/2012/10/03/jon-stewart-slams-v...


I can buy that voter fraud is pretty limited. But it's simply untrue that there's not a single example.

People are convicted and go to jail for it. Not many, but some. Saying they don't seriously undermines the credibility of the argument.


That's true, the more careful version of the argument is that it's in such minute quantities that it could not possibly swing elections. In addition, to the extent any voter fraud does happen, it's extremely rare for it to be of the kind where voter ID would solve the problem, i.e. people voting under false identities. The most common kinds of voter fraud are people with perfectly legitimate documents, e.g. voting in both your old and new residence in the same election, under your real name with a real ID in both cases. Or, registering in a jurisdiction that isn't actually your primary residence.

Some numbers I can find for Texas: they convict about 5 people/year for voter fraud, of which 0.3/year are of the false-identity variety. (http://abcnews.go.com/Politics/OTUS/voter-fraud-real-rare/st...)


Allocating massive resources to stamping out a statistically insignificant issue (given that we have no intelligence on who those votes are for, the end result is moot) makes it hard to take Republican voter ID frenzy at face value.

Those strategists are smart and they know exactly what happens if you make it harder for the down-trodden to vote.

Moderate, intelligent Republicans should be embarrassed by Voter ID. Is this the small government you keep referring to?


Philly turnout in 2008 was around 60% of registered voters, and seems to be roughly the same this year.

There is a lot of conspiracy theory going around, but not much solid evidence of irregular voting outside noise levels. Especially true because of how closely the outcome matched polls: it seems unlikely that these mythical ballot-stuffers were also able to somehow rig all the telephone polls over the past few months, so that the eventual results would match.


ok so reality check...

Philly county has 644,768 total votes currently with close to 100% reporting.

10% rather than 8% of that total is (generously) 65,000

Current lead for Obama in PA? 284,000 votes

So even if we amplify your 8% claim to 10% and over-round it is still not even close to making a difference.

Also if you are going to make claims about such contentious things it would be nice to cite some source.

My numbers above come from google's election results site

http://www.google.com/elections/ed/us/results


Not that I agree with patrickzgill's evidence-less assertions, but in his scenario, the fabricated votes wouldn't be the percentage over 100%, but the percentage over the real turnout rate (which might be 70% or so).


That's true. I was assuming he meant 8% OVER expected turnout, since 108% turnout is frankly ridiculous. That's one of the problems when people don't cite anything we have to assume what the numbers mean.

Taking a more nuanced view, one way the numbers could be true is by people voting in specific precincts that they don't live in. I.E. they go to the polling place near work not home. Which would just mean that one PA vote is counted in a different part of the state, not actually affecting the outcome.

States have different rules governing this kind of thing and I have no idea what the laws in PA are like.


WOW I am re-reading almost word for word a thread that I read during Bush v. Gore 2000.


Ahh that's Philly for ya:

http://www.foxnews.com/politics/2012/11/06/judge-issuing-ord...

Unfortunately, it happens everywhere and from both sides.


I'm confused on how the point about fundraising is relevant to your other points.


Compare the hyperbole in the article, breathlessly talking about the geniuses on the campaign - then compare with the hard numbers compiled in the link I posted.


The amount of money raised is not the only metric for measuring how smart of successful a campaign is.

I don't think it's even a good metric.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: