Hacker News new | past | comments | ask | show | jobs | submit login
How Visa Predicts Divorce (thedailybeast.com)
96 points by subbu on April 8, 2010 | hide | past | favorite | 48 comments



Slightly off-topic: if you post something with a misleading title that you can improve, please do so. This article has no information at all on how Visa predicts divorce, just the third-hand assertion that they do. An Amazon text search within Supercrunchers yields nothing of value, either.


Exactly. Why is the word "how" in the title? Here's a choice quote from the article regarding that question:

"Exactly how the credit industry does it—through sophisticated data-mining techniques—is a closely guarded secret. "

Well great! Now the title isn't simply misleading, it's factually inaccurate.


"Why is the word 'how' in the title?"

Because TDB raise tens of millions of dollars so they could hire dozens of celebrities and go on Colbert the day they launched, and they're trying to make their money back.


I've heard rumors that visa uses its purchase records as a leading indicator to trade in stocks. Having all of these sales records gives them a leading indicator of how a particular retailer is performing, before the company releases any statements, or speaks with analysts. While the concept sounded simple, I'm not sure if I believe this because I had no way of knowing how large of an advantage it would translate into. Has anyone else here heard these kinds of stories about visa or banks? Is there are reason why this kind of thing would be illegal or ineffective?


I know of one financial firm that did this. They didn't trade the specific stocks, but they used their credit card data to predict what the Fed would do.

Doing it one stock at a time might raise some eyebrows; it sounds close to insider trading. But it's accepted that people who bet on interest rates will gather lots of data--since every economic indicator you can imagine will have some effect on rates, it's both impossible to be a true "insider" and impossible not to have some degree of insider information.


> Doing it one stock at a time might raise some eyebrows; it sounds close to insider trading.

That's not the definition of "insider trading". Insider trading is you trading on your company's info - the company is free to do so. For example, Google employees can't trade Rackspace on knowledge that Google is about to make a big deal with Rackspace. Google, on the other hand, is free to do so.

From http://en.wikipedia.org/wiki/Insider_trading

"In the United States and many other jurisdictions, however, "insiders" are not just limited to corporate officials and major shareholders where illegal insider trading is concerned, but can include any individual who trades shares based on material non-public information in violation of some duty of trust. This duty may be imputed; for example, in many jurisdictions, in cases of where a corporate insider "tips" a friend about non-public information likely to have an effect on the company's share price, the duty the corporate insider owes the company is now imputed to the friend and the friend violates a duty to the company if he or she trades on the basis of this information."


The definition of insider trading is alarmingly flexible; it's safer to avoid the appearance of impropriety, since that can still get you convicted.

I wasn't aware that a corporation could trade based on insider knowledge. That seems very unlikely; it would imply that, e.g. Boesky was busted because his fund was organized as a limited partnership. And wouldn't a corporation with one owner be equivalent to that owner in terms of economic interest? Could I just form byrneseyeview Inc. to take advantage of my insider knowledge?

Do you have a source for the statement that corporations can engage in insider trading?


I think you misunderstood his point. Any entity is allowed to trade on information that's legally available to them without violating any duty of trust -- if for example, Google decides to buy rackspace, it is perfectly legal for them to buy call options in front of the announcement to profit from the spike in value caused by the announcement, as no duty of trust is broken.

It would be, however, illegal for employees, shareholders or indeed anyone hearing about Google's intentions about the purchase to buy those options, as they would be breaking their duty of trust to Google by doing so. This holds regardless of the legal status of the entity in question -- a corporate shareholder getting privileged information from Google before the announcement would be just as screwed as an employee.


I assumed that a deal with Rackspace meant a purchase of Rackspace products. I agree that you can buy a stock before trying to buy out the company--but you can do that as an individual, too.

But in this case, the company trading on credit card data is clearly analogous to a Google employee, not to Google. They have material information about the prospects of another company, and they use that information to trade.


> I assumed that a deal with Rackspace meant a purchase of Rackspace products. I agree that you can buy a stock before trying to buy out the company--but you can do that as an individual, too.

It doesn't matter what kind of transaction Google and Rackspace are doing - it's illegal for Google employees to trade on this information and legal for Google to do so. In other words, Google employees can't trade on news of a Google deal to buy Rackspace the company or Rackspace equipment/services. Google the company is free to trade in both circumstances.

> But in this case, the company trading on credit card data is clearly analogous to a Google employee, not to Google. They have material information about the prospects of another company, and they use that information to trade.

Not at all. Insider trading is defined as improper use of corporate information by employees. Companies are free to act on their information.

It doesn't make sense to define "insider trading" as use of "improper information", let alone "non public information". Remember - a huge fraction of trades are based on non-public information.

For example, I may think that the iPad is a disaster. That's not public information, yet it doesn't make sense to restrict my ability to trade on it.


> But you can do that as an individual, too.

If you are a secretary working for Google, and overhear talks about the buyout and proceed to load yourself on the stock, SEC is going to want to have a word with you.

> I assumed that a deal with Rackspace meant a purchase of Rackspace products.

I don't think this would be illegal either -- with whom would Google break it's duty of trust if they bought half of Rackspace just before announcing a large deal?

> They have material information about the prospects of another company, and they use that information to trade.

I have material information of the prospects of every company I trade -- I wouldn't trade them without. What turns this into illegal insider trading is receiving that information from a source that is bound by duty of trust to not speak about it. As long as the material information about the prospects of a company does not originate from the company in question, or anyone else bound to not divulge it, it's not insider trading.

When visa data mines credit card transactions, the information so gained can be freely used to whatever purpose without fear from insider trading accusations.

That being said, I do not think it's moral or right -- the laws just were written before large-scale data mining was feasible.


If you're an individual, and you plan on buying a company, you can accumulate stock in it first without doing insider trading. That's what I mean when I say you can do that as an individual, too.

with whom would Google break it's duty of trust if they bought half of Rackspace just before announcing a large deal?

This may be accurate. I was under the impression that insider trading referred to possessing and using information that would have a material effect on the stock price. i.e. nobody's "duty of trust" is breached if a cab driver overhears his rider talking about a big deal, but I'd assume that the cab driver could get in trouble for acting on the information. If it's really about trading based on information obtained through a breach of trust, you end up in an odd situation: either any breach makes all trades insider trades (i.e. executive tells information to his secretary, who leaks it to a journalist) or any non-duty-bound link in the chain obviates the trust issue (X tells it to Y who tells it to Z--who is overheard by A, who sells the tip to B, who makes the trade.)


> If you're an individual, and you plan on buying a company, you can accumulate stock in it first without doing insider trading. That's what I mean when I say you can do that as an individual, too.

Yes, but that's because it's your information that you're buying said company.

However, there are other rules that apply to corporate take-overs. In the US, you have to disclose when you reach a certain ownership level. (And no, you can't avoid that by colluding with other people.)

> This may be accurate. I was under the impression that insider trading referred to possessing and using information that would have a material effect on the stock price.

That's why I posted the definition and link.

> i.e. nobody's "duty of trust" is breached if a cab driver overhears his rider talking about a big deal, but I'd assume that the cab driver could get in trouble for acting on the information.

Actually, the person who was overheard violated the breach of trust.

> If it's really about trading based on information obtained through a breach of trust, you end up in an odd situation: either any breach makes all trades insider trades (i.e. executive tells information to his secretary, who leaks it to a journalist)

Huh? There's at least one obvious breach of trust in that chain.

Yes, any breach makes the resulting trades insider trades. No, that doesn't imply that all trades involve such breaches.


whoa. insanely cool. Given the market penetration that visa has, they'd get data that's fairly representative of the US (or world?) spending habits and i'd assume it'd be HUGELY effective. I doubt that they do it, and if that became public knowledge I have to imagine they'd come under some serious fire, but I don't think its illegal.

The notion of running a credit card company basically for free as a way to get information about the economy and made MUCH more profitable trades in the financial markets really appeals to me.

Friend of mine has been working on using google trends/blogs as such an indicator, but its a lot harder to tease tradable information out of that sort of data.


Yeah it sounds cool - I don't know that this is real just a rumor. A lot of things that sound cool turn out to have serious problems.

I saw something on here a while about a "marketplace for data" - I can't remember what it was called - but it make me wonder how one would determine the price for a data feed or open source intelligence - it seems that a fair price for a data stream should be how much of a profit you can make by trading against people who don't have access to the data.

But it seems to me that if monitoring revenues of retailers was a really big advantage you could do almost as well by conducting a small well designed survey like the FED does. VISA's volume of data is probably deep into large N range of the sigma/sqrt(N) sampling error.


As actual moves on retail chains are probably quite small on a quarter-by-quarter basis, to be able to make any kind of money on this kind of information you have to be levered up to your ears. Which also means that if you are wrong, you are going to lose a lot of money. Getting that error probability way way down is a priority.


And if you think your purchases say a lot about you, just wait until somebody gets a hold of your social network data.

(I am willing to bet that "Percentage of friends who divorced" and "Percentage of friends who divorced recently" both whallop any naive demographic predictor of divorce risk you care to name.)


Rapleaf is already making progress with this.

http://www.rapleaf.com/apidoc/person


The comments on that article are telling. The loss of privacy due to inappropriate data mining looks like it may cause at least some people to "go Amish" slowly (now, before I get jumped, I just mean "the avoidance of technology" by "go Amish"; please don't get religious over that).

Personally, reading this article makes me glad I never opened a FaceBook account. I wish gmail was not so all-knowing either; I hate seeing very personally directed ads (anti-depressants when I am sad after reading a break-up mail, for example). Just too creepy.


Anybody that breaks up using email in stead of a face-to-face is probably not worth being sad over.

How people part ways says lots about them.


Does Visa actually know what you've bought?

"Cardholders who purchased carbon-monoxide detectors, premium birdseed, and felt pads for the bottoms of their chair legs rarely missed a payment."

My bank statements only state where something was bought, not what was bought. I'm not aware of a store that only sells felt pads for bottoms of chairs and premium birdseed.


That statement was referring just to Canadian Tire, a retail store that also offers credit cards. I assume the card company can only get that kind of detailed information if you make your Canadian Tire purchase with your Canadian Tire MasterCard.

I once purchased a CO detector at Canadian Tire... maybe I'll bring that up when I apply for my next credit card?


The article is badly titled. It explicitly states after the lede that the author was unable to determine how Visa predicts divorce.


The Underground History of American Education posits that the main factor at work in the school system is the homogenization of people for the benefit of big business. From a tech perspective, it's easy to understand this motivation; wouldn't it be easier to develop that killer mobile coupon app if everyone had a device with the same capabilities?

Now I'm beginning to wonder if data mining is making the homogenization of people obsolete, if hundreds of profiles can be created, and it suffices business interests to simply be able to categorize you?


They don't only want to homogenize the consumer, they want to also homogenize the workers. A fungible workforce is a boon.


Off-topic, but related to stuff mentioned in the article.

How do we relate the idea that we own the data a company like Visa collects about us? For example, our buying habits should be "our data." While I understand the need for privacy, people go on to worry about how Visa will use this information, and it's impact on privacy.

Does Visa have the right to use this data to target users?

On one hand, we'd be highly upset if Visa didn't use the data it collects in certain cases, like anti-fraud and tracking odd behavior and warning us of it. But then you have an article like this asking for consumer protection.

I guess the issue here is one of balance: who owns the data? If it's the user, does the company really have the right to use that data to provide a service with it? Even if it's anti-fraud or simple customer service? This isn't something new, either. It's only become easier and easier to do more accurately as technology grows.

On one hand, privacy is important for obvious reasons. On the other hand, generated data is useful and can be used for improving our lives. Just as privacy protects the criminals as well as the innocent, generated data can be used for ill or for good.

I don't think I'm asking anything new. I've always been a person that wants to track everything users do. On sites I create, I want to track everything, know everything. Yes, I want to do this so that I can increase profit. But it's only because I believe that a user who attempts to make a purchase and fails really wanted the product, and it's my job to do everything in my power to get that product to him. I have no desire to steal his data, sell his information, or anything else. But I can see how something like this could be misrepresented or misused.

Just some thoughts I had on this. I'd like to hear other points of view on both sides of the spectrum. I admit I don't know enough, and while my motives are honest, they are in a way profit driven. Customer services is profit driven (Apple doesn't provide the CS it does because it loses money on it!)! And if anyone has any good links on the topic, please share.


"does the company really have the right to use that data to provide a service with it?"

Whenever I've signed up for a card, it's had a checkbox for whether or not the company is allowed to send you "great offers from our affiliates". Presumably, they'll only use your data if you allow them to. (Whether they hold to that in practice is another matter.)


But that has more to do with affiliates. Can Visa do what they want within Visa with your data?

Let's say Google for example, tracking your search habits. Now it uses that data to modify what it displays to you, like ads. This is all internal. They don't release it to the public, or even advertisers.

Even Visa. While it can't release your data, can it use that data to send you information?


>Does Visa have the right to use this data to target users?

Its probably buried somewhere in legalese in one of the Product Diclosure Statements.

I was surprised by some of the things my bank had wanted me to agree too several months ago when they "updated" this document, so much so that I switched banks.


Factoid: The author, Nicholas Ciarelli, was also known as Nick de Plume -- founder of thinksecret.com.


I highly recommend Super Crunchers to anyone who has not yet read it. It is written for the layman, but nonetheless it is highly fascinating and insightful for anyone who is remotely interested in data mining.


I found the book very disappointing because it glossed over any real detail.


Like I said, its "written for the layman". If you want to get into high detail for one of the many examples in his book...you will eventually find yourself at places like this: http://www.retrosheet.org/game.htm :)

Also if you want some more technical detail about the overall process there are plenty of books out there: http://news.ycombinator.com/item?id=1055042.


According to the article UPS predicts when you are high risk of leaving their service and will offer a sales call. I assume the salesman is allowed to offer perk/discounts to get a customer to try to stay. I'm wondering if these models can be figured out so you can intentionally trigger something like this to get deals from companies that do data mining.


The mobile social network Loopt or its competitors could conceivably predict with 90 percent accuracy where an individual will be tomorrow.

I would think that Visa would be better positioned to do this than Loopt, because people are more compelled to use their credit cards when they go somewhere. You can't go to a store and forget to pay, although you can forget to checkin.


Correction: The headline of this article originally referenced MasterCard, not Visa.

As is evident from the post URL. But why the change?


Another headline correction that is needed is that the article doesn't tell us how Visa predicts divorce, just that they can.


Since they don't seem to tell what their success rate & the accuracy of the predicted date are, I wouldn't say they "can" do it, which would imply that they are 100% successful at predicting the day of a divorce.


On the left it says:

> How Visa Predicts Divorce

> by Nicholas Ciarelli


Captain Obvious at work? How does it answer the question? Assuming you know what the question was about?


Political parties have been doing this for some time. Check out http://www.voterlistsonline.com/version2/site/page.asp?page_...

You can learn a lot about a potential voter (ie is he worth making a stop when canvasing?)

Purchase Premium Fields, including:

    - Home purchase/mortgage/refinance information
    - Presence/ages of children per household
    - Magazine subscriptions
    - Hunting/fishing licenses
    - Charity contributions
    - Additional polling options - including Targeted Random Sampling and Normalized Sampling*


I wonder how expensive it is to purchase things that decrease your risk rating, while also using cash for things considered particularly risky.


You have to be more careful than that.

You need to write some script to follow random links on HN while you're away from your computer, so when you're off doing risky things you won't change your profile.

You need to occasionally forget your cell phone, so when you're doing risky things, forgetting your cell phone won't change your profile.

If you have a car, you need to pass through the intersections with red light cameras at the normal times, but no extra times, so you don't change your profile.

Does visa buy dns lookup histories? or cellphone active tower histories? or car intersection histories? probably not. yet. better to build the habits now though.


Stepwise linear regression is how they do it. All statistical packages marketed since the late 1970's include the procedures. It's what the AI craze of the early 1990's used in their black box apps.

Basically you associate a narrow set of outcomes (Filing for divorce, filing for bankruptcy, reporting a stolen credit card) and a mountain of possible predictors (store type, store name, store location, number of purchases from store in the billing cycle, ranges of purchases from a given store within the cycle). The stat program builds a regression equation out of the raw data by literally predicting trying to predict the past from data in the further past.

The cool thing is that you can't put in too many variables (as in you don't have to know if something may be related or not) since the uncorrelated variables will simply vanish in the final equation.


Even better would be to have a probabilistic weight to these various items. Computers are way better at finding patterns and irregularities in patterns than we are at making irregularities. If I were to "accidentally" forget my phone based on my own decision, it would probably cluster in some way, say, forgetting it every 17 days +/- 1, or on Thursdays more than any other day, or something.

So create a profile, then use RNG to match yourself to that profile.


Given that the article does not actually talk about how Visa predicts divorce, maybe we can speculate. Here are some ideas:

* Meals at two restaurants on same night.

* Separate vacations.

* Ages approach mid-life.

* Card used on personals websites.


I doubt there is a rationale trend. It's just statistics, really. My guess is that it has more to do with the amount of money spent by each spouse rather than what.


It's probably pretty easy, because in addition to what is being bought, Visa knows where the purchases occur. Most people have two clusters for their purchases: work and home. A person who develops a third cluster may be having an affair. If you know what's being bought, you can figure it out with even higher probability.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: