In his paper, Kosinski said the computer was zeroing in on certain physical features of the face consistent with prenatal hormone theory (PHT), which suggests that gay people are so because they were exposed to differing levels of androgens in the womb. “The faces of gay men and lesbians had gender-atypical features,” the scientist wrote, “as predicted by the PHT.” Kosinski argues that prenatal hormone theory is “widely accepted” as a model for the origin of homosexuality.
Did they control for the actual contents of the photos? For the position and expression on the faces? Nope!
Yet even Kosinski admitted that the computer might be picking up something besides immutable facial features. His algorithm, for example, posits that gay men are more likely to wear glasses. “Many wondered why faces with glasses are considered by algorithm to be more likely to be gay,” Kosinski said. “It might be something else in the face that’s also correlated with having glasses.”
This sounds like a classic case of the "Neural Net Tank Urban Legend", which was discussed here recently [0].
Jonathan Frankle, a Ph.D. student at MIT who served as the staff technologist at Georgetown University’s Center on Privacy and Technology, said we often don’t know how these sorts of algorithms work. Unlike ordinary code, a neural net involves nodes that are interconnected, with processes happening in parallel. A neural net isn’t just executing a set of instructions in sequence; instead the nodes are talking to each other, giving feedback to each other. There really isn’t any way to trace how it makes its decisions — one can’t look at a single line of code or subroutine. It’s effectively a black box.
I hate this. It's wrong. You absolutely can trace the execution of a neural network in its decision making. It might be hard to understand, but there's nothing stopping you from looking at node activation patterns.
There is also a whole class of techniques for extracting some kind of meaning out of these black-box-type classifiers. See LIME [1] and older technique called "partial dependence" [2].
Reminds me of the anime "Psycho Pass". It's set in the not-so-distant future where everyone is monitored and assigned a "crime coefficient". Without giving too much away, the system starts to show its weaknesses when a criminal is able to murder people and commit crimes without being detected by the system.
The shape of an eyebrow does not indicate an extrovert, at least not with any better accuracy, than a Human interviewer can achieve with five minutes of conversation. And then there is the question whether introvertedness is even a disqualifying trait in a sales representative. A good team will probably have both introverts and extroverts at some ratio.
Yes, it's "bad science". But it's also bad business. It's the same with genetics: As soon as we have a full understanding of risk factors and beneficial traits, and we're far from that, people competing for a job would probably all have multiple "bad" and multiple "good" qualities, which cancel out much of the benefits of testing in the first place.
Did they control for the actual contents of the photos? For the position and expression on the faces? Nope!
Yet even Kosinski admitted that the computer might be picking up something besides immutable facial features. His algorithm, for example, posits that gay men are more likely to wear glasses. “Many wondered why faces with glasses are considered by algorithm to be more likely to be gay,” Kosinski said. “It might be something else in the face that’s also correlated with having glasses.”
This sounds like a classic case of the "Neural Net Tank Urban Legend", which was discussed here recently [0].
[0]: https://news.ycombinator.com/item?id=15485538
--------------
Jonathan Frankle, a Ph.D. student at MIT who served as the staff technologist at Georgetown University’s Center on Privacy and Technology, said we often don’t know how these sorts of algorithms work. Unlike ordinary code, a neural net involves nodes that are interconnected, with processes happening in parallel. A neural net isn’t just executing a set of instructions in sequence; instead the nodes are talking to each other, giving feedback to each other. There really isn’t any way to trace how it makes its decisions — one can’t look at a single line of code or subroutine. It’s effectively a black box.
I hate this. It's wrong. You absolutely can trace the execution of a neural network in its decision making. It might be hard to understand, but there's nothing stopping you from looking at node activation patterns.
There is also a whole class of techniques for extracting some kind of meaning out of these black-box-type classifiers. See LIME [1] and older technique called "partial dependence" [2].
[1]: https://github.com/marcotcr/lime
[2]: https://github.com/bgreenwell/pdp