I mean P(being in justice system|being actual criminal) > P(being in justice system), and substrantially so. Moreover, P(being criminal|being in justice system) > P(being criminal). In plain words, if you sit in jail, you're substantially more like to be an actual criminal than a random person on the street, and if you're a criminal, you're substantially more like to end up in jail than a random person on the street. That's what I see as causal relationship. Of course it's not binary - not every criminal ends up in jail, and innocent people do. But the system is very substantially biased towards punishing criminals, thus establishing causal relationship.
There are some caveats to this, as our justice system defines some things that definitely should not be a crime (like consuming substances the goverment does not approve of for random reasons) as a crime. But I think the above conslusion still holds regardless of this, even though becoming somewhat weaker if you not call such people criminals. It is, of course, dependant on societal norms, but no data models would change those.
> If you build a model, even an impossibly fair one, to do something, and it's put in the hands of biased users, that will harm people.
That is certainly possible. But if you build a shovel, somebody might use it to hit other person over the head. You can't prevent misuse of any technology. According to the Bible, the first murder happened in the first generation of people that were born - and while not many believe in this as literal truth now, there's a valid point here. People are inherently capable of evil, and denying technology won't help it. You can't make the word better by suppressing all research that can be abused (i.e. all research at all). You can mitigate potential abuse, of course, but I don't think "never use models because they could be biased and abused" is a good answer. "Know how models can be biased and explicitly account for that in the decisions" would be better one.
> his[0] read as incredibly condescending, which doesn't help.
Didn't read condescending to me. Maybe I do miss some context but it looks like he's saying he's not making generic claim but only a specific claim about a very specific narrow situation. Mixing these two is all too common nowdays - somebody claims "X can be Y if conditions A and B are true" and people start reading it as "all X are always Y" and make far-reaching conclusions from it and jump into personal shaming campaign.
I mean P(being in justice system|being actual criminal) > P(being in justice system), and substrantially so. Moreover, P(being criminal|being in justice system) > P(being criminal). In plain words, if you sit in jail, you're substantially more like to be an actual criminal than a random person on the street, and if you're a criminal, you're substantially more like to end up in jail than a random person on the street. That's what I see as causal relationship. Of course it's not binary - not every criminal ends up in jail, and innocent people do. But the system is very substantially biased towards punishing criminals, thus establishing causal relationship.
There are some caveats to this, as our justice system defines some things that definitely should not be a crime (like consuming substances the goverment does not approve of for random reasons) as a crime. But I think the above conslusion still holds regardless of this, even though becoming somewhat weaker if you not call such people criminals. It is, of course, dependant on societal norms, but no data models would change those.
> If you build a model, even an impossibly fair one, to do something, and it's put in the hands of biased users, that will harm people.
That is certainly possible. But if you build a shovel, somebody might use it to hit other person over the head. You can't prevent misuse of any technology. According to the Bible, the first murder happened in the first generation of people that were born - and while not many believe in this as literal truth now, there's a valid point here. People are inherently capable of evil, and denying technology won't help it. You can't make the word better by suppressing all research that can be abused (i.e. all research at all). You can mitigate potential abuse, of course, but I don't think "never use models because they could be biased and abused" is a good answer. "Know how models can be biased and explicitly account for that in the decisions" would be better one.
> his[0] read as incredibly condescending, which doesn't help.
Didn't read condescending to me. Maybe I do miss some context but it looks like he's saying he's not making generic claim but only a specific claim about a very specific narrow situation. Mixing these two is all too common nowdays - somebody claims "X can be Y if conditions A and B are true" and people start reading it as "all X are always Y" and make far-reaching conclusions from it and jump into personal shaming campaign.