The points that this article and many of the comments here seem to make about the dangers of these racially-caused mistakes miss the more fundamental problem of this technology being used ever more widely in the first place. Facial recognition will invariably get better (virtually certainly to the point of almost completely avoiding mistakes based on a persons ethnic makeup) and that's what's really terrible, because it hints at the sort of constant surveillance future we have coming toward us. If anything, it's almost comforting to know that right now, at the very least, these algorithms don't yet work well enough to avoid politically unfortunate errors, because the political consequences of these errors slow down the scenario of total, nearly error-free ubiquity that anyone who even slightly values privacy should be very worried about.
The inherent problem facial recognition and similar systems have is that they just rely on an extremely narrow context. But I agree that it is not constructive to points towards bias (in many discussions actually), since that will just normalize the presence of the technology itself. Suddenly it just needs to solve the problem of improving some statistics instead of justifying the deployment of cameras everywhere in the first place. So the criticism is a particularly bad one that omits to ask the hard questions.
I would agree if these systems weren't being used to accost and prosecute people in the affected groups in the real world today. Considering that humans are willing to blindly follow these false results I'd say it's even more frightening rather than comforting.
The article seems to hang on the false positives being higher for certain minorities, but makes no mention of the false negatives being higher as well.
So while some ethnic minorities might be more likely to match incorrectly to a known person of interest, they're also more likely to be let through if they are indeed that person. I think that the first case is definitely more damaging on the whole, but I still find it misleading to not mention the specific scope of the "racial bias."
it doesn't seem relevant. In a country like the US the law pursues criminal individuals, not criminal groups. The fact that the system also falsely negatively identified people doesn't somehow 'balance out' false positives, it's not like the point of these systems is to randomly catch 100 African-Americans. (although given the political ideologies of some of the people in the surveillance tech space maybe it is)
The fact that it produces errors in both directions just makes the system even worse in total.
The reason it's relevant is that false positives are always a trade off against false negatives, so the "easiest" way to reduce false positives is to increase false negatives. But false negatives are also very bad, so we can't really do that or we're just trading one problem for another instead of actually solving anything.
The only real solution is to improve the overall accuracy of the system, but that's easier said than done. Some of the main sources of the inaccuracy are intrinsic. The population in question is a smaller proportion of the general population so there is less training data available. People with darker skin absorb rather than reflect light, which makes it harder for the system to identify them.
So the problem isn't some racist schmuck who just needs to be fired and the system will get more accurate, it's a consequence of demographics and physics. There may be a solution somebody can find, but there isn't guaranteed to be, and in the meantime there's not a whole lot you can do other than trying to find one.
The principles of the law makes it clear that it's far, far worse to have an innocent person in jail that a guilty person go free. Many legal principles revolve around this.
> The principles of the law makes it clear that it's far, far worse to have an innocent person in jail that a guilty person go free. Many legal principles revolve around this.
Yes, absolutely. But there are two issues in this context.
The first is that facial recognition isn't a conviction. Presumably if it identifies the wrong person then they'll have the opportunity to demonstrate who they are, e.g. by showing that the name on their ID is not the name of the suspect, before being convicted of the crime.
And the second is that ordinary application of that principle is incompatible with eliminating the disparity, because if you shift in favor of false negatives in general then it reduces the false positive rate for all races but the proportions stay the same. But if you shift it only for one race then you're only making the existing racial disparity in false negatives even worse, which is hardly fair to minority communities because it implies they'll have a higher proportion of criminals on the streets victimizing their communities.
That being said, in this context having a preference for false negatives over false positives in general probably is a good idea, and it does reduce the absolute (rather than proportional) racial disparity in false positives -- which is not nothing. But it's the proportional disparity that people are always complaining about.
That has exceedingly not been the case in recent time. Consider the existence of plea deals which punish people for insisting they be proven guilty in a court of law.
Are they? Isn't our entire justice system premised on the idea that false negatives are far more acceptable than false positives? There's a problem when your false negatives are systematically racially biased, but beyond that, false negatives seem like what you'd expect and desire from such a system.
Why not change the training data to reflect an equal amount of the population? Officially the US has six “races” according to the Census Bureau. So, another option is to make each race ~16% of the training data.
Some might object to the decreased accuracy this gives whites — but if we’re going to have any hope of preventing existing systemic inequalities from being encoded in AI systems, we’ll have to socially engineer the data sets.
Because now the system is only going to perform at peak accuracy in some fantasy world split up into 6 races of equal part man, woman, child, and elderly. If you’re seeking a realistic model, you randomly sample actual data.
There are many problems with socially engineering data sets. For one we would need to achieve consensus on what constitutes an appropriately representative (i.e. socially just) data set. And then we need a feedback loop to make sure that our dynamically changing inputs don’t knock our meticulously tuned output out of whack. And we’d need to continually adjust our model based on our changing definition of social justice. Finally we need to find social engineers with credentials in building perfect societies (I most certainly don’t trust Joe PM) to perform all this work.
Sociologists have been arguing forever that humans are inherently racist. Maybe these “AI systems” are just proving their point. Perhaps this bias is more a reflection of our inability to maintain global context that is also maximally relevant within our local tribes. Maybe we should focus on acknowledging biases and working through them rather than trying to transcend into some unicorn world where everybody has all the context all the time and bias is impossible.
Anyway, I see your census and raise you an FBI: white people commit way more crimes in the US [1]. So if your goal is to reduce total harm then social engineering is not your solution. Perhaps more accurately identifying the most frequent perpetrators (in the case of the US, white people) is exactly how you’d end up tuning your system...
It doesn't contract the 13/50 stat at all. And also says that "white people commit way more crimes in the US".
Specifically: 4,209 (48.7%) "murders and nonnegligent manslaughter" ("murders") are categorized under race=Black. Incidentally, still less than race=White (4,261 or 49.3%).
Indeed, 12.6% of respondents in the 2010 census self-identified as "Black or African American".
That's your 13/50 meme.
However, murders are <0.085% of all crimes tallied in the FBI report, and are far outweighed by crimes like larceny (1M instances), drug abuse violations (1.27M), DUIs (1M), and "Other" (2.9M), where race=White are all by far the majority.
Thus, 69.4% of crimes are committed by race=White, so "white people commit way more crimes in the US" is also true.
Because per capita only matters when you're comparing groups to each other, not when you're trying to minimize the overall number of failures in the system.
Dude we’re talking about identifying general criminal suspects with facial recognition. Either you’re trolling or you might actually be racist. Your answer does not “stand” up anything other than a point that black people per capita in the US commit more murders than white people—which nobody is disputing or even talking about. The facial recognition we are talking about is used to pick out general suspects from a “crowd” (which is why I used arrests). We are not trying to identify per-capita murder hotspots and we most certainly don’t need facial recognition to do so. Your argument is ultimately pointless which is why you’ve been venerably downvoted.
> Why not change the training data to reflect an equal amount of the population? Officially the US has six “races” according to the Census Bureau. So, another option is to make each race ~16% of the training data.
The difficulty with this is that you commonly don't have any control over the training data. Someone else collected it and you can use it but you can't choose what's in it.
And in some cases you already have census-level data, which means you couldn't even increase the number of people with lower representation in the data set by collecting more data, because there are no such people not already in the data set. The only way to have equal representation would be to throw out data you do have on the groups with higher representation. Which is obviously objectionable because we need to be improving accuracy for the group with lower accuracy (if we can), not reducing accuracy for the group with greater accuracy just to make the numbers the same.
This attitude is one of the many reasons why we can't have nice things. Instead of trying to elevate everyone to best possible standing you want to reduce all to the worst one.
>"Some might object to the decreased accuracy this gives whites — but if we’re going to have any hope of preventing existing systemic inequalities from being encoded in AI systems, we’ll have to socially engineer the data sets. "
So to fix systemic inequalities for some groups, we're going to increase systemic inequality for certain other groups?
Definitely a step in the wrong direction as it's just going to swing hatred back the other way. And N-levels later of this "fiddling" you'll end up in a spot where neither side can reconcile with the other because each side has legitimate reasons to feel slighted and no one can look past that "hatred". Just look at the mess that is currently in the middle-east after people "fiddled", and then "fiddled" again to try fix it.
> So to fix systemic inequalities for some groups, we're going to increase systemic inequality for certain other groups?
No; in order to accurately identify people through appearances, we'll try to sample evenly throughout the range of appearances and features instead of sampling based on proportions within the population of the people setting up the system.
You're can't be seriously making the case that white people are such an overwhelming portion of the world's population that not concentrating on them in your training data constitutes racial discrimination. Not only racial discrimination even, but is a attack that will inspire a "legitimate" counter-retaliation.
No that's not what I'm saying. If you look at the quote I mentioned, the person was suggesting we "socially engineer" data sets in order to "preventing existing systemic inequalities from being encoded in AI systems".
This has wider implications and tacit-meanings that I think I picked up on and definitely implies doing more than just having an equal sampling of each racial group. How that would be enacted, I don't know. The point is to not swing the pendulum the other way in order to "fix things" as it'll cause problems down the line, but to rather just "fix things" in the most non-intrusive and fair way.
I'm glad you feel this way. So do you agree that it is therefore unacceptable to use such a system even if one accepts the surveillance state that comes with them since it demonstrably disadvantages minorities and making them "fair" would only make things worse? Or is this concern more selective?
I'm pretty libertarian, so I lean to simple solutions and my view on "fairness" is very non-complicated. I.e. Yes there may be bias in this technology we use, however, I would say that the important part we should really be focusing on is "how" that tech gets used, not the fact that it isn't perfect in some "unfair" kind of way. All sorts of things around us are unfair to all sorts of sub-groups of people that I don't think we should be going down this path and mixing government intervention in order to resolve it.
Yes, it's kinda unfair on some level that the algorithms used don't work so well on women/blacks/children/elderly as well as "white middle-aged men", but it's not unfair in the "government should do something about it" kind of way just yet. If someone gets misidentified, that doesn't mean police should just go out and "arrest" them. Rather we add a person into the mix and have them talk to that person, make a final call. That is actionable counter-case data that we can use to resolve the bias in the training sets.
A lot of facial-recognition systems include human components in order to resolve errors. E.g. I've worked with facial-recognition software here in South Africa that was developed by Asian companies where the majority of their data-sets and tests were using Asian individuals, so one has to manage error risk using humans in the loop.
Depends on what kind of bias you want/need to manage? Corruption/bribery is a big risk factor in SA so that is usually the one most people worry about, rather than racial bias.
One of the things you have to look at is "how" that bias may manifest and if bias can even be applied in any negative sense in the realm of control you give to the user. If you give someone a task where they are to compare two faces, and they don't like the demographic of that individual, I suppose they might pick incorrectly (because you only gave them true/false type options). But you can build in review processes, reporting, performance feedback, etc to mitigate that. You need not treat it as bias at all, merely bad performance/mistakes that need correcting.
Consider a hypothetical scenario in modern police work: First pass returns a false positive on suspect match, police act on the false positive in a way that results in jail or death due to racial bias. This rarely manifests as a performance issue with easy remediation.
I'm not sure adding more humans to the mix is a solution to solving bias problems because humans are unconsciously biased and may be perversely incentivized.
Increasing overall accuracy won't help if it's applied to all ethnicities. It has to be an increase only for the affected type of people. But if you're going to do that, you could also just artificially cripple the accuracy for the too-accurate groups. That sounds like the easiest solution, if it's actually a problem to be solved in the first place.
No, it would help a lot. Let's say you shrink mistakes from 10% and 20% to 1% and 2%. That means the gap in misidentification likelihood has shrunk from 10% to 1%.
Huh? Isn't that just clever accounting? 2% is still twice as much as 1%. If a black person and a white person each travel the same number of times, the black can still expect twice as many additional checks as the white, the same as before the improvement.
No, it’s the appropriate measure of ”unfairness.” If you designed a system where 5 black guys got misidentified but 1 white guy, out of the U.S. population, that would be far less unfair compared to 2%/1% despite the ratio being higher. And why did you pick that ratio to compare instead of 90%/80% vs. 99%/98%? The right means of comparison — for fairness — is the absolute difference in proportion.
I don’t mean to imply that’s the right metric at all, of course, because it specially privileges minorities. The proper measure should equally weight the algorithm’s performance on all people in the population, or at least weight them irrespective of race. That’s just not a measure of fairness between races.
>The fact that the system also falsely negatively identified people doesn't somehow 'balance out' false positives, it's not like the point of these systems is to randomly catch 100 African-Americans.
If they're disproportionatly committing crimes, skin color becomes a strong prior. It would be rational, iff the goal is to lower crime rates. But I don't know that it is morally OK.
Also I'm not sure if over representation is the source of bias in this particular case, haven't read the article yet.
Edit: yeah, this article is just a smear, wapo is salivating over anything they think confirms that police are racially biased. It doesn't mention anything technical about the training data or typical nets being used. I'm not advocating for facial recognition tech, I'm personally against it, but racial bias coming from objective data is the least of your potential worries.
And what people also don't realize is machine learning system will get massively more accurate over time, and its accuracy landscape will generally follow the distribution of the training set. Academic training sets this is typically bootstrapped on are very different from what you'd encounter in the US criminal justice system, but as the system is fed real data it can get crazy good at what it does. Superhuman accuracy facial recognition systems have existed for a while now. Humans just aren't that good at matching faces, and it's not like humans are bias-free either.
It seriously depends on the technology being used. There isn't any evidence from human psychology that dark-skinned people have harder-to-recognize faces, so we're probably just using the wrong approach in the technology currently available.
Part of the technology is the capturing of photons, fewer of which bounce off dark skin than light skin, so it is in part the same generic problem as always: fewer photons, less data, worse images. Whether it's day vs. night, fast lens vs. slow, close vs. distant, fast moving vs slow (requiring fast shutter than lets in less light), or whatever causes it, fewer photons mean more problems.
And even if you get enough photons to, for example, recognize both black and white equally well in daylight, you might not at night or in a deep shadow indoors, or up close equally well but there will be some distance at which the advantage of more photons begins to matter, or some speed, or some combination of factors.
Any technological improvements will help, but there won't be any technological solution that will always work equally well with fewer photons as with more.
Eyes are just better at distinguishing similar dark colors against each other compared to cameras. Only the most expensive, massive and heavy cameras (costing hundreds of thousands of dollars, generally) can compete with eyes in this respect.
Plus human facial recognition is massively overpowered for normal cases. I think if you tested it in challenge cases (through blurry video, or snow, or in the dark, or at a distance), you might see darker skin tones being recognized less easily. It'd be an interesting experiment.
That might very well be true, but if it is, then it re-enforces the point: the technology may not be where it needs to be yet, and if it isn't, the tool probably shouldn't be deployed in public use cases where the end-result could be inherently discriminatory by race.
The very best HDR cameras have roughly the same dynamic ranges as the human eye.
You can't display all of this range to people because of the limitations of display technology, but you can feed the full range to a facial recognition engine.
Security budgets might not stretch to top-end HDR equipment but the price keeps on coming down. The performance of a modern flagship phone is remarkable compared to a few years ago - and fixed surveillance cameras can have much bigger glass and sensors, making it cheaper to get super-human performance.
One new but related issue is that body-worn cameras can capture more low-light detail than the human eye. Police unions have argued against deploying these sensors, because, they want the evidential record to show what the officer could see - not what a cat could see.
Less reflected light, all else being equal, means the same amount of detected noise (whether using cameras or eyes), and therefore a worse signal to noise ratio on the dark image. Worse signal to noise ratio means less information. Less information indicates worse detection accuracy.
While there isn't evidence that dark-skinned people have harder-to-recognize faces, there is also no evidence to the contrary.
It seems like the null hypothesis in this case, given no additional evidence, is to assume that darker images are indeed harder to recognize.
What study makes you think it is significant for human faces? I'm noting absence of evidence; if you're making the point that it doesn't imply evidence of absence, I agree, but I've also never heard anyone seriously claim "black people's faces are just harder to recognize." The anecdotal claim isn't even present.
The linked study on cars is analogous at best, but doesn't prove anything for human facial recognition. Cars on highways and human beings in various settings are extremely different circumstances.
To a degree this comes down to how you capture an image of their skin. Lighting has a big impact on how the result looks as does the medium you use to capture the image.
In the early days of color photography there was an issue with some films and reference images being tuned for the most common subject (light-skinned humans) and as a result if you tried to capture a mix of races you'd get bad results: https://petapixel.com/2015/09/19/heres-a-look-at-how-color-f... This makes sense if you consider how light is (generalizing here) a broad spectrum of hues and a given material is most reflective for specific parts of the spectrum, so if you don't capture much there you'll get a low-contrast image, like stripping the R channel out of an RGB bitmap.
It's of course possible to solve the problem for a wider set of skin tones, and it has been solved, but it takes more work. It's a subject of ongoing discussion/experimentation in film to this day: https://www.konbini.com/en/cinema/insecure-cinematographer-h...
Well the consensus from the article is that middle aged white men had the best rate of success with facial recognition which sounds like a data mismatch problem to me.
Complicated answer. Theoretically no, I was just mentioning highlights vs shadows on the face which are properties of reflectivity. The backdrop certainly matters from a camera exposure perspective in order to optimize for the skin properties in play.
the point is that these products would not have shipped at all if their workforces were diverse enough to point out “hey this doesn't work, like at all” or “hey this glaring edge case isn't far enough on the edge to send this to production”
That's a bold assumption. Such algorithms are developed wwith training sets, not just by employees randomly clicking around. They wrote that 189 algorithms were submitted, there were surely a few developed by non-racist, diverse people.
It's the methods being used to select, populate, label, and validate the training set that are the problem.
Basically, your team is composed of white dudes who don't see the problem with a ML training set consisting largely of pictures of white dudes.
To prevent this they'd have needed to A) Employ a black person, and B) Listen to said employee's feedback, in order to recognize the problem.
Edit: Also worth pointing out, just using a representative population sampling would still show racial bias, essentially weighting accuracy with respect to population percent. You'd probably need to have equal samplings of pictures of people from all races/genders/disabilities if you wanted equal accuracy across the board. That also includes picture quality and range of picture quality. Doubling up images, or using corporate headshot white dudes and grainy selfie People of Color could still cause issues.
Same logic applies to labelling. That minimum wage contracting firm used to decide who's who in the photos may exhibit racial bias, by virtue of the fact that most people do. If their accuracy in labelling is racially biased then so too will the algorithms that it's based on.
Why do you need A to do B? Obviously any competent ML practitioner can notice a biased training set. Discriminating against people based on an assumption that race determines ability is just racism.
Are you asking why you can't listen to an employee that doesn't exist?
But more to your point, if these oh-so-competent ML practitioners were doing their jobs right, we wouldn't be having this discussion.
The whole reason diversity is championed in hiring is precisely because a single individual's perspective can only see so far. And if you have a monoculture team who has experienced very similar life circumstances, you end up with the kind of narrow perspective that leads to more racist soap dispensers.
I think I agree with your specific point concerning facial recognition, but not your general point about un-diverse teams being incapable of delivering a good product for a diverse audience. This is because I have recently worked with a team that spent considerable effort on accessibility issues despite none of the team having any disability.
I'm glad that you were able to deliver a product that helped with accessibility, but part of the considerable effort it took was just in trying to understand what someone else's perspective is. It's less efficient than simply having a member of the team with real lived experience that can answer common-sense questions that your team agonized over answering.
And not to say this happened in your case, but even with that considerable effort, it's still very easy to end up with blind spots in your product that a more diverse team would have caught.
It's the same as hiring for any other level of experience for more routine technical skills. If your team has no experience in this area, they'd need to expend a much greater degree of effort to answer questions that someone who is experienced would already have known the answer to.
I agree. I just have low expectations as far as diversity in tech goes (so, one black person and/or one woman on the team would still be a milestone many of these companies have yet to reach), but you make a good point: hiring only one person of color or one woman would still be very problematic.
Not a number per se, but ideally the racial and gender ratios of the team would reflect the total population of the area (country? Metropolitan area?) in which the company resides, with the standard caveats that random sampling would give a range of different combinations. So that'd be, a multinomial distribution with the probabilities that each race is chosen set at the demographic percentage of the total population.
In other words, ideally the racial and gender distribution of a team would be as inconsequential and unbiased as blood type or handedness, in that the aggregate demographic ratios on your teams would at least match that of the residential population in your area, and ideally that of your broader geographic location.
I'm not doing a good job explaining this clearly, but the simple answer is: more than one. No one wants to be the token hire.
> In other words, ideally the racial and gender distribution of a team would be as inconsequential and unbiased as blood type or handedness, in that the aggregate demographic ratios on your teams would at least match that of the residential population in your area, and ideally that of your broader geographic location.
Ok, I don't know about race, but for gender look up the "gender equality paradox". In countries with greater equality rights for women they show less of an interest in STEM subjects.
Like I say I don't know of any similar studies done for race, but it would indicate that you shouldn't necessarily expect outcomes that "would match that of the residential population in your area, and ideally that of your broader geographic location".
In my opinion we should be pushing for equality of opportunity, not equality of outcome (you appear to want the latter).
That's why I sprinkled the word "ideally" so liberally throughout that comment, as that was the premise of your question. To actually reflect the population, you'd have to somehow address centuries of institutionalized disenfranchisement, de facto segregation, educational barriers, racially-moticated policing and disproportionate conviction rates (highlighted by the OP). That's a lot to put on a hiring manager who isn't even sure that racism is an actual, tangible thing.
Realistically, the most that hiring managers (save for huge FAANG institutions) can do is thoroughly ensure that their team isn't inadvertently (or blatantly) racist/sexist in their hiring process and on the job, and to post the job in enough places that a diverse applicant pool will see the posting.
With that said, hand-waving away that there are few to no women or African Americans/Latinos/Native Americans/etc. on the team with an overzealous application of the Equality Paradox is a pretty dangerous mindset to get into. It's essentially passing the buck, and is eerily reminiscent of the claims made by 1950's Southern US Politicians that Blacks were the ones who were self-segregating because they wanted to, not the other way around.
What I'm saying is, the ideal 50/50 gender ratio/representative race may be unrealistic for a myriad of reasons, but if you're a 50-person start-up with 2 women, one of whom is HR, and no black people, I'd take a good, hard look at the company culture that's being fostered, and particularly whether turnover for women and People of Color at your company is worse than average.
> Realistically, the what most hiring managers (save for huge FAANG institutions) can do is thoroughly ensure that their team isn't inadvertently (or blatantly) racist/sexist in their hiring process and on the job, and to post the job in enough places that a diverse applicant pool will see the posting.
I have been involved in hiring people before. There was absolutely nothing racist or sexist in the way we hire. Fact was we go two applicants. Neither were women or minority status. Fact is that the industry is full of white men (even here in Europe).
We should be hiring on ability to do the job and nothing else.
So, a quick note that my comments regarding racial representation are somewhat US-centric. I'd expect a company in Europe to be a much higher % white because your population is.
> We should be hiring on ability to do the job and nothing else.
This is exactly my point! Yet there is quite a lot of inadvertent, or even blatant, racism and sexism that happens during the hiring process and on the job.
Because programming ability isn't determined by melanin concentration in the skin, or by what pronouns you use, and hence shouldn't show selection bias in hiring, save for a biased selection process.
In the same way that handedness has no bearing on programming ability. If your firm were staffed entirely by left-handed folk, it should rightfully raise questions about how that happened.
You seem overly defensive about your hiring process, and unsatisfied with any answer that asserts men and women are equal. Ironic, no?
the hyperbole is not really helpful or substantive, its not about the algorithm its about how the product wouldnt have shipped when individual contributors, their manager, the director, the executive team, the board, and their vendors all noticed that it didnt work very well when the demo failed because they were not white males testing it on themselves and their colleagues and photos of their friends
The technology will inevitably improve. Real issues will arise when facial-recognition becomes perfectly accurate (without false positives, racial bias, etc). Governments will be able to track citizens anywhere.
On the other hand, some governments may find it extremely convenient to use 'accidentally' biased technology to harass and persecute members of disliked minorities.
Right now? No. Could it? Sure. Minorities being 'accidentally' targeted for harassment in 'coincidental' yet statistically reliable ways is something many governments have a long history of doing.
Doesn't this also prove, by proxy, that this fact can potentially (if not certainly) apply to other AI & associated fields?
Forgive my ignorance, but this seems to lend credence to the popular idpol claim that the white guys programming AI are ignorant of the inherent bias their models might have.
You are assuming that AI's are only programmed by white men, and you are assuming that it hasn't occurred to anyone who is an expert in the field of facial recognition that their algorithm's might be used with people other than white people.
These are two bold assumptions. Would you really have us believe that this is the case? I think you will find that some important information has been left off. The article suggests that asian created algorithms are better at recognising asians. It implies that they all have problems with darker skin tones. Why does the article not explore the reasons why? Perhaps they are wanting to say that algorithms can be racist, rather than the truth of the matter, which is there are certain technical issues that are difficult to overcome with darker skin tones.
An article about the technicallities of cameras and facial recognition on darker skin tones would generate a lot less clicks than "Computers are racist!"
I think that "less accuracy for a certain race" + "facial recognition used for finding criminals" is where the word bias (notably "bias" and not "racism") comes from.
The first sentence of the article backs up its usage of the word bias instead of inaccuracy:
> "...casting new doubts on a rapidly expanding investigative technique widely used by law enforcement across the United States." emphasis mine
Any heuristic will have varying effects with respect to different groups of individuals, regardless of what property individuals are being grouped by.
It's absolutely irrelevant whether the heuristic is unfair to individuals of a particular race, or unfair to individuals of a particular nose length. Prioritizing the former over the latter is a totally arbitrary value system that deprioritizes the most important metric for maximizing fairness, which is the overall accuracy rate.
Ultimately any inaccuracy of the heuristic is an instance of the heuristic treating someone unfairly. The objective should be to minimize the inaccuracy rate overall, not the inaccuracy rate in relation to politically prioritized groupings like race.
it looks to me like they buried the lede in the last paragraph.
'relationship “between an algorithm’s performance and the data used to train it,”'
To solve this problem they should look at the relationship between the number of pictures for a given race, and the accuracy in recognizing members of that race. It probably does best at identifying European descendant faces because it was developed in a country where European descendants are the largest ethnic group. I'd bet dollars to donuts that if you can increase the number of faces for each race this disparity will largely disappear.
In China, you will heard of the company SenseTime who specialize in facial-recognition. I bet if you do the same study on that system, you will see the same racial bias on white people! All the problem is data. Try train on man faces and apply it on women faces. If you want to solve this problem, gather more training data on those "minorities" -- even if that means you need to go to a foreign country to collect them.
"Algorithms developed in Asian countries had smaller differences in error rates between white and Asian faces, suggesting a relationship “between an algorithm’s performance and the data used to train it,”
Quick, somebody offer the guy who realized this a job.
Surprise surprise, in a field dominated by white men, the technology is deemed production-ready when it’s accurate for white men.
Not blaming this group or anyone in particular – we all just need the be cognizant of the fact that we have our blind spots (racial, gender, and otherwise).
Of course, accurate data is hard to come by, but this report from Reveal from The Center for Investigative Reporting suggests otherwise, especially at executive levels:
The article casts doubt on expanded use of facial recognition but that’s not the right reaction IMO. Facial recognition systems are used as a first pass filter. That is they are reducing a large space of possible matches to a lesser space of possible matches. It is significantly better than expecting beat cops to match faces on the sidewalk as they look around, for example. No one, to my knowledge, is relying on facial recognition systems as absolute matches.
The ACLU rep that is quoted is IMO relying on hyperbole to make his point (“One false match can lead to missed flights, lengthy interrogations, tense police encounters, false arrests, or worse”). If the match is close enough that a human would confirm it as well, and conduct an “interrogation”, then presumably facial recognition technology did not incrementally add to the problem of mismatching since a human would also make the same false match.
So if it makes policing more efficient and enables more criminals to be nabbed, and it is used as a first pass filter (so the false positive rate doesn’t matter), I am all for it.
> If the match is close enough that a human would confirm it as well, and conduct an “interrogation”, then presumably facial recognition technology did not incrementally add to the problem of mismatching since a human would also make the same false match.
it seems likely to me that cops might use it as justification for a fishing expedition, or TSA agents may just go along with it as a cover-your-ass measure
