If I were training a classifier to predict whether a sentence is talking about household activities v/s not, wouldn't the occurrence of man/woman in the sentence be a good feature? Today, woman do perform household activities more (whether we like it or not), and wouldn't it make sense to use that piece of information when performing some predictive analysis?
The technical sense of "bias" arises when the train and test distributions differ. Obviously if you train with a dataset of text from a foreign country's news and then apply it on an American context, the difference in the data distributions will introduce bias, but why do we need a social twist to this already well-functioning term? If the same classifier is trained and evaluated in India (with its sexist roles, say), then there's no (technical) bias and I don't see why it's a bad application.
Can you give an example of a situation where an ML application would be reinforcing a problematic bias but still have good performance metrics? My point is that a wrongly-applied ML application would suffer in just plain accuracy. For instance, a Automatic Carrier Counsellor might give "homemaker" as a suggested career choice to women, but then before we start calling it biased, it would already be wrong. If the same algorithm had dug deeper, it would have learn that the said woman would be a great programmer.
Recidivism prediction systems will usually tell you that black people are more likely to get arrested/convicted again. They do so accurately, but also result in longer sentences for black people.
Yeah but doesn't that have more to do with the way the predictions are used?
It seems to me to be a stupid thing to do. This person seems more likely to get convicted again, lock 'em up longer. Instead of asking why is this person more likely to get convicted again? Can we prevent this in a redemptive non punitive way?
It's really useful to have that prediction/data but how you use it is more important
the problem is a layperson doesn't necessarily know what a prediction necessarily means without a deep understanding of how the system is making its predictions, let alone how to apply it.
worse is that since the prediction is coming from computer that lends the prediction an air of authority another article called "bias laundering". the general belief is that computers are objective and cannot have bias, which in a sense is true, but people don't tend to think a step further about the problems and biases in the people who programmed the computer.
so that is definitely a thing usually missing from these discussions is that the people using these systems generally don't know how they work, and believe they predict or imply things that they don't
I mean, that same algorithm could be used to determine that blacks or other at-risk groups should receive extra attention or support. An accurate picture of reality can be used poorly or well.
There is a much more fundamental problem, which is that people are bad at understanding the difference between "is" and "should". No amount of information about what the world looks like tells you anything about what course of action is the most moral (and vice versa). If you are building a system that predicts recidivism rates (figuring out what "is"), then any piece of information that improves your accuracy is good. If you use that system to suggest sentences (making decisions about "should), then you are going to run in to a lot of problems.
I still don't understand why information should be elided from judges during sentencing. If public officials use data to worsen issues like recidivism rather than improve them, then those officials should be removed. If a judge can't be trusted to act responsibly and morally with accurate information about a defendant then why would we even begin to think that they're competent?
It's just the reality of how the justice system works. We have trust in the approximation of justice that the judiciary provides and constantly struggle to improve that judiciary.
I think you have a really good point here. The problem is that we have this current bias in society and people wish to change it. I think there is a fear, that if we reflect this bias, in the way we talk, we re-enforce the bias.
It seems an effective tool, if you want to change thinking then police the way words can be used around the topic. It is however worrying that machines could start playing a role in this. It could become a powerful tool in steering public opinion. This doesn't seem too bad, but that could be used to favour an incumbent political party, or more than likely to sell products we otherwise don't really want.
But you are right machines need accuracy and removing that bias could be detrimental to the task they're solving.
That's fine if you are measuring the bias component too. If you're not, you risk perpetuating it. It's natural if I read a sentence about hands to conclude that the test refers to people and not fish, but if I read about 'the hands of a surgeon' and assume that those hands are male based on the current demographics of the surgical profession, I'm making an unwarranted assumption on insufficient information. It's wise to grant utility to uncertainty by maintaining a 'Don't know' option rather than being in a rush to make a determination before it is necessary, not least because of the computational cost of unwinding incorrect assumptions.
Totally agree with you. I'm not at all trying to say accuracy and bias can be orthogonal.
I'm trying to say some people think they have a good enough reason to throw away accuracy if that means they can change a societal bias. But that can only be a good thing if you agree with the change being made.
Isn't it preferable to accurately measure and account for Bias? The same metrics can be used to combat bias that reinforce it. Without accurate metrics, how do you form your argument against bias? Feelings?
No, it would not be a good feature. For one thing, baking the bias of existing practices as opposed to constraints risks reinforcing that practice as more and more decision-making is left to ML. Second it makes your system vulnerable to verbal paradoxes designed to exploit that bias.
This is the tug-o-war of influencer v. influencee. A machine that just tells-it-as-it-is might hold an advantage over one that willingly ignores some data to promote a different view of the world.
Personally, I see more danger in people trying to make machines that evangelize their own biases to the world than machines being molded by the existing social assumptions of society, given that we expect machines to perform most of the work/control most of the resources in the future.
If you are going to "debias" your model, what is the point of even training the model to handle these issues in the first place? Not surprisingly, human language can be biased. If you train a model on human language it will not magically transcend those biases. The problem is that people have this expectation that ML is going to lead to these perfect decision makers.
Machine Learning creates models that reflect the data, not the truth.
In the sjw-religion, why is "homemaker" considered inferior to "computer programmer"? One of the oldest and most important human occupations versus hunched over at a desk slaving for a salary until being outsourced to a bot in 5 years? I've never understood the default sjw/"feminism" assumptions that anything feminine is "bad".
It isn't so much bad or negative as it is risky. A homemaker, male or female, becomes financially reliant on his/her partner. A partner who can die, become too disabled/ill to work, or who can leave after the homemaker has missed his/her key career/skill building years.
Women are more likely to be pushed or encouraged to take this important supportive role to benefit others while putting themselves at risk.
It goes both ways, the partner can become reliant on this kind of support. And only sometimes partly you can cover the difference with money and/or power - typically much more expensive than cooperation.
In fact I'd say the housekeeping skills are always more marketable, more basic and easier to master - you can live off them, but not without. This gives rise to competition which drives both social perception of value and actual financial value down.
Mutual dependency has been the way of life for ages - for good reason.
Edit: Thought police strikes again! Instead of downvoting, please provide a coherent argument why a given point is invalid or how.
It is a lot easier to hire someone to take care of housekeeping duties or to do them yourself than it is to make up for 10+ years of little to zero non-homemaking skill development.
Not as easy as you'd expect if you want someone actually competent and versatile.
You would be surprised how big a bag "homemaking" is, ranging from cooking, through teaching, down to clerical work, through basic finances and back up with handyman (yes yes) fixes. Any of those skills specialized in is marketable, though not respectable on its own.
Note how few of them are true knowledge and research work.
All of them are in the areas where there have been major reductions in number of jobs due to automation and centralization.
Because it pays less (therefore enforcing financial gender inequality for which there is no good reason) and has been stereotyped for longer. It is sort of backwards thinking.
The often unspoken assumption is that stereotypes are strictly bad and evil. Without stereotypes, all the social conduct just breaks down and explodes - people suddenly become unpredictable.
However, stereotypes (including SJW stereotype) can cause big frictions between groups. Even more so when they're actually inaccurate, invalid or misapplied.
Another thing is something called "stereotype threat" which reinforces certain behaviors while punishing other - a kind of self fulfilling prophecy at times. You think you would behave as if some label would be applied to you therefore you behave to fit in. The drive to fit in is human, social and often subconscious.
How does it pay less? You get to have a place to live, eat, sleep, all for free, without having to stay away from your children for an extended part of the day.
To whom is the paycheck made out? In a relationship with a large pay disparity, there is a financial dependence and vast power difference between the involved parties.
If your partner started to abuse you, would you rather be the homemaker or the person whose name is on the paychecks?
For the individual spouse there may be some disadvantage (at least in terms of risk) to being the homemaker. But for the aggregate of family households it's not so clear. A previous HN comment speculated that much of the economic gain from increased female workforce participation - or more accurately double-income households - flows to capital, as it increases both labor supply and overall consumer demand, leading to downward pressure on wages and upward pressure on price of consumer goods (plus huge demand for child care). By contrast the value delivered by a stay-at-home spouse (of either gender) flows largely to the family. Not a very PC analysis for either the "left" or the "right". Wish I could find the original HN comment, or see further research.
I understand what you're saying and agree with you in general, but I think you're missing a large part of the picture. In general, women have far better options for ending a marriage than men - as of 2006, 3.6% of alimony recipients were men, for example.
> If your partner started to abuse you, would you rather be the homemaker or the person whose name is on the paychecks?
In that circumstance, I'd rather be a woman. While it's very difficult to find anything resembling an unbiased source, it seems apparent that it is substantially more difficult for a man to seek law enforcement intervention as a victim of domestic abuse.
> In general, women have far better options for ending a marriage than men - as of 2006, 3.6% of alimony recipients were men, for example.
Isn't that in large part because of the employment and income disparities between men and women, though? If women are employed less often and generally make less when they do, it makes sense that they're more often the recipient of alimony.
> In that circumstance, I'd rather be a woman. While it's very difficult to find anything resembling an unbiased source, it seems apparent that it is substantially more difficult for a man to seek law enforcement intervention as a victim of domestic abuse.
My comment was actually only regarding homemaking vs. computer programming, not how those occupations are gendered. The parent said "How does it pay less? You get to have a place to live, eat, sleep, all for free" — all I meant was that if there's e.g. domestic abuse, all else being equal, it's far better to be the partner who actually makes money than the partner who relies on the other's income.
At work you are also a slave to your boss, in a sense. If your boss started to abuse you, wouldn't you rather be a home maker so that your partner deals with the day to day shittiness of having to be in a job?
The way to not be afraid of abuse it's to have a support network. For example, extended family, friends, neighbors, acquaintances, etc. This applies to you whether you're an employee or a homemaker.
As a human you are always dependant on other people. Part of being a responsible adult is figuring out how to cooperate with people.
> The way to not be afraid of abuse it's to have a support network.
Your support network is significantly more flexible if you have access to money. You haven't (and I suspect can't) made any real argument that there's not a vast power imbalance in a relationship where one party makes the most or all of the money.
It's not that anything feminine is "bad", it's that women are often pressured into doing "feminine" things because reasons.
There's no reason "homemaker" should be inferior to "computer programmer", but there's also no reason that homemaker should be "feminine". Men are equally capable of taking care of the house, and women are equally capable of programming computers.
Women get pregnant; men don't. Women have to nurse children (as in breast feed), men don't. When a woman it's pregnant or nursing, she needs support. That's the whole reason why there is such a thing as marriage. Because you can't raise children on your own.
> Women get pregnant; men don't. Women have to nurse children (as in breast feed), men don't. When a woman it's pregnant or nursing, she needs support.
Absolutely none of this is relevant to whether non-women or women "should" be homemakers. Women get pregnant but that doesn't mean they're in any way more suited to taking care of children than non-women.
> Because you can't raise children on your own.
Tell that to the literally millions of single parents who do a great job raising kids while millions of two parent households fail at it.
Except companies that outsource such services and the whole service industry in hotels or such. Use less straw in your strawmen.
Of course this is often parasitic and I suspect you meant directly. Even then, thing called "gold digging" has a long long tradition. (Though being misattributed to women while it works on both sides just slightly differently. Often with more than a hint of slut-shaming etc.)
Edit: excellent. Call a thing like it is, get downvoted. People jump to conclusions, proving the point.
The technical sense of "bias" arises when the train and test distributions differ. Obviously if you train with a dataset of text from a foreign country's news and then apply it on an American context, the difference in the data distributions will introduce bias, but why do we need a social twist to this already well-functioning term? If the same classifier is trained and evaluated in India (with its sexist roles, say), then there's no (technical) bias and I don't see why it's a bad application.