Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Why do we need metrics at all?

If I’m sat in Einstein’s office doing a performance review with him, I’d probably say “Yes, Alf, you’ve done pretty well these last few years. Keep it up.” Don’t think I’d need a metric. And if he came to me for funding for a postdoc I’d say - yes, sure.

I don’t know when we decided that _management_ had to be replaced with _metrics_ but it doesn’t seem like a good idea.

Management involves doing things that don’t scale, that’s why we have lots of managers.



This leads to another problem that we (the science community) are already facing: how should we allocate funds between older and more established researchers vs new researchers? Or, who will be the next Einsteins?

If you only fund the more established researchers, then new researchers are starved out and more likely to leave the field. However, my belief is that new researchers are more likely to develop big new innovations than (say) the really old professors that are well past their prime.

You also have another related problem: given a large pool of new researchers, who are the ones that will be really good? Plus, there are other possible goals, like spreading the money around to broaden the base of researchers.


> how should we allocate funds between older and more established researchers vs new researchers?

Reasonable question with no precise answer, but I imagine a manager would seek a balance between the two as with any company or team. Some big hitters but you've got to see where your next Einsteins are coming from.

There's nothing about this that is solved by metrics. Metrics just help you might shallow decisions quickly, and provide ways for academics to game the system by manipulating those metrics.


Einstein solved this problem by doing his best work before joining academia, and winning lifetime funding as a reward afterward.


Einstein was not peer reviewed either. Peer review became standard only later.

That is to say, whatever worked for him or that era wont work for contemporary person today.


Probably because it's much easier judging whether to fund Einstein than whether to fund someone much further down the chain.


I don't see how that is relevant, but let's consider someone further down the chain.

"Ah yes, Jimmy Postdoc, I see you have published 3 papers with an average impact factor of 3.1. Have a promotion."

vs

"Ah yes, Jimmy Postdoc, I see that you're making progress in improving quantum error correction, as evidenced by the fact that we can now use 80% of the previously required qubits to complete Shor's factoring assuming a surface code - great work, you should get that paper out at some point but keep focused on the work for now.

I'm also really pleased with your contribution to the academic community in the dept, particularly helping out Polly PhD student with her SAT formulation of decoding. The constructive questions in her talk really opened up a new line of enquiry. Great job.

Given all the above, have a promotion."

Metrics are stupid, people are smart, stop using stupid.


The subjective valuation lead to quite a lot of nepotism and unfairness. There is techno-babble you can use to pick pretty much anyone if you are good enough with words.


I agree that's a problem of traditional management. The way to counter it is good management methods, in particular ensuring that reviews and promotions are cross-validated using other personnel, both horizontally and vertically. I'm sure you're right though, some bias will slip through.

The question you have to ask yourself is whether you are prepared to tolerate occasional suboptimal decisions for a metrics based evaluation that _corrupts the entire system_.


> I’d probably say “Yes, Alf, you’ve done pretty well these last few years. Keep it up.”

That might work in a company but not in the academic world. Many countries limit the amount of time a PhD student or Postdoc researcher can stay at a university. After that time the person has to find a permanent contract (professor) if they want to stay. Because there are many many more candidates than available positions, the hiring committees try to justify their decisions by objective (haha) criteria.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: