> People are starting to realize that this 'digital reputation' could limit their opportunities. (And that these algorithms are often biased, and built on poor data.)
That's interesting of itself, but the bigger underlying issue is that opportunities are becoming more concentrated. When only a few companies dominate hiring in many fields, their mistakes get seriously amplified. Back in the day you were fine if Google's hiring process misjudged you - you could work for Excite or Altavista instead. Nowadays if some ML algo decides that people wearing blue sneakers are worse job performers you can get screwed (without even knowing why). And even worse, the major companies (where the jobs are) often share algorithms.
Worse still-- past employment at the majors is seen as a strong "social proof" indicator by many including other employers and investors.
I saw an angel.co drinking game once. I think you had to chug two drinks for "worked at X" (where X is any major) being put forward as the sole qualification of a founder or key early employee. This is starting to edge out "went to X" where X is a top-tier school.
China is supposedly deploying their own horrific state-sponsored "social credit score" system, but we're doing it too. We're just doing it in a less centralized way. In a way that's worse. In China everyone will know of this system and its existence and I'm sure people will figure out so many ways to game it it'll become irrelevant. In the West people will remain blissfully ignorant as ours has no name or formal identity.
Ultimately I am still more creeped out by what our private sector is doing than what our NSA and CIA are doing. Neither is good, but the latter has some oversight and regulation. The former has absolutely no regulation or oversight whatsoever, and in any case the private sector is very often better at such things than the public sector is. I wouldn't be at all surprised if Facebook's data analytics are far superior to the NSA's.
One reason why I think management in the US is so bad is because the entire corpus of managers went to the same few groupthink institutions and that is why they are so bad--no real risk takers, no real anything. It's like that in politics, too. We aren't promoting the right ideals, but merely conformity, risk-aversion, and foregone conclusions accepted as fact.
That's interesting of itself, but the bigger underlying issue is that opportunities are becoming more concentrated. When only a few companies dominate hiring in many fields, their mistakes get seriously amplified. Back in the day you were fine if Google's hiring process misjudged you - you could work for Excite or Altavista instead. Nowadays if some ML algo decides that people wearing blue sneakers are worse job performers you can get screwed (without even knowing why). And even worse, the major companies (where the jobs are) often share algorithms.