Hacker News new | past | comments | ask | show | jobs | submit login

Instead of downvoting you I thought I'd ask, how likely do you think people do see you gaming and don't call you on it? I've been in many more situations where you don't have the power or ability to call someone out on something because it's more complicated than just "I'll tell on you".



It's your boss not being fooled by your gaming and not promoting you.

Its not something you report.

Like taking a long lunch break.

People aren't blind, they can see what you are doing, people will talk about it behind your back.

Goodharts law relies on a mythical assumption everyone in the organisation is just out to scam it.

It's all measured in real life. If people are evil and out to game metrics why won't they just game the system when there are no metrics?

A good version of Goodharts law might be, people will work towards metrics so some times it's worth tweaking metrics to reduce incorrect outcomes, but sometimes worrying about the inefficiency is worse than the inefficiency itself.


I think your last paragraph gets to the point.

Goodhart’s law is not about people “gaming” the system. Sure that could be one of the effects of aligning incentives to metrics, but it’s really about about the dangers of aligning _goals_ to metrics (of course incentives are often tied to achieving goals).

For most non-trivial systems it can be difficult to correctly identify and describe all of the variables that affect the system and in what ways. For example, at a high level a company’s singular goal may be something as abstract as “make more money”. It’s pretty difficult to just _do_ that, so we break it down into smaller, more concrete goals that we feel will contribute to the above. How? Good news! Last quarter we hired a business analyst who started recording detailed metrics about our business and processes. Now we can simply look at the metrics from last quarter and try to improve upon some key numbers.

And this is where Goodhart’s law comes in. You see last quarter we were not _trying_ to improve upon any specific numbers. Although we can see that we achieved poor efficiency in our inventory processes last quarter, it turns out that focusing on improving this metric actually net lost us money because it created more inefficiency in shipping!

No one has to be “gaming” the system for the above to occur. A system full of good actors can fall prey to turning metrics into goals (at the expense of the system as a whole). Goodhart’s law is just a warning. Of course we still want to measure things!


If it isn't a goal (or a subgoal) then why measure it? Groups have many goals but what usually happens is that certain goals are prioritized.

Unfortunately, because of incentives, the anointed goals (or the metrics) get over prioritized causing worsening externalities until the goals or metrics get adjusted to correct the externalities. "Move fast and break things" turns into "Move fast with stable infrastructure".

Of course this is a dialectic, so expect the goals and metrics to shift again as new externalities are exploited. The ultimate correcting mechanism is a god who isn't a slave to the incentive structure and who resets the system by getting rid of bad actors.

The world is the Matrix.


> If it isn't a goal (or a subgoal) then why measure it?

Often the goals are hard to measure, so you measure a proxy. This can work well if people are actively focused on the goal, and check the measure to see how they're doing. It stops working if people forget that the measure isn't the goal, and work to increase the measure.


Your boss has the most to game from you cheating. He’s probably the reason you are gaming the metics in the first place. And if you do it sloppily he will fire you.


If everybody's gaming something a little bit, nobody is incentivized to take action on most things, even if someone reports it.

It often takes egregious abuses to get people to care.

(And is this so bad? The end result of "perfectly efficient" is pretty brutal for the individual worker.)




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: