I haaate these numerical ranking systems. Bank of America had one when I worked there. Managers would have meetings with each other to match their workers with ranks. Really, really long meetings. So long that my boss apologized to me after the meeting to discuss my ranking.. because heleft before my name came up and there was no followup meeting. I ended up being ranked towards the bottom, I didn't get a raise, and the bonus was meh.
When I left, they told me that my performance was great and that they wanted to give me a substantial raise, since apparently, my pay was far below what it should be. They then had to replace my weekend shift with four others for a while. Most of the team was gone a bit later.
The sad thing is that the managers participating in these sorts of activities are unable to connect cause & effect.
For example, if you rank individual employees then you destroy teamwork and incentivise all sorts of behavior that's bad for the company. It hurts morale, lowers productivity, increases employee turnover, increases hiring costs, lowers product quality, etc. It's a vicious cycle. And for someone who thinks it's a good idea to rank employees, these effects will simply confirm their beliefs that they need to rank employees.
Why does this happen? In the best case, it's a misguided attempt at career development. Or it's a way to figure out who to give more money and who to fire. Usually there is an element of fear: "I'm afraid my employees will not work hard unless there is the threat of a bad performance review".
To be fair, ranking employees does sound like a good idea if you don't know anything about humans. Most management activities are really just things that sound like a good idea (but fall apart under scrutiny). To be more fair, bad management is the norm, so no one has good mentors and role models. "If everyone else does this, then it can't be wrong." It's another vicious cycle.
I would like to make your point clearer because I think there isn't sufficient visibility of this problem in our community.
The problem here is 'stack ranking' or 'curve fitting'. The assumption behind this model is that the quality of workers you have will mirror a bell curve ( for example, 10% will be poor, 80% will be average and 10% will be excellent ). The management tries to identify these individuals and assign them ratings based on this assumption. Your promotion, bonus and pay hike is based on these ratings. The problem with this approach is that no matter how well you perform there are only 10% 'very high' ratings available. This means that if someone else succeeds, then the chances for your success has diminished. This creates an incentive for you to sabotage others at work and pitches team mates against one another.
I worked at a place that stack-ranked, and we didn't have this problem. It was typically pretty clear and well-accepted (by both leadership and peers) that high performers were in fact the best people.
In a big enough company, you can't sabotage other people (because you might not work closely with them) and it's pretty transparent when you do so.
The problems I saw were threefold:
1) Bad Managers - Far and away #1. The example of a manager leaving the ratings meeting comes to mind. Conversely, a manager who just has it out for you: that's the sabotage that I saw on more than one occasion (but that's not exclusive to stack ranking). [Edit: also worth noting that word got around and bad managers were avoided, so the system eventually would self-correct.]
3) Comparing Stretch Roles - Pure performance is tough. I never thought we gave enough credence to great people doing good in tough roles vs. good people doing great in easy roles.
3) Comparing Different Impact - We got to the point where we were trying to compare developers with business analysts. The former would have outcomes like "developed 75 test cases" and the latter would be "convinced the CEO to invest $1M". It didn't help that the business managers were better at quantifying outcomes than IT managers. (See #1.)
You can easily sabotage people in large companies, probably more easily than in small. Just don't offer support when you could. Start gossiping. Gang up with one or more other people and make it seem like the other isn't a team player.
I would go as far to say it is easier to do in a large place as you have more of a buffer between you and actual work. So then you have more time to concentrate on politics.
Those actions are the same at every level of company. I think it's harder at big companies because you don't have the opportunity to offer (or withhold) support or create perceptions about people you literally might never come in contact with.
If you're doing that to someone on your team, (good) managers will notice the disconnect.
Look, is it possible? Of course: in every environment. The person who's going to be unethical and sabotage others is going to do so regardless of whether stack-ranking is involved.
> The person who's going to be unethical and sabotage others is going to do so regardless of whether stack-ranking is involved.
The trick is not to think about people as purely ethical or unethical. Imagine, if you would be willing to tell a small child, right before an arithmetic exam, 1 + 1 = 3 for $10 billion. Will you do it ? The harm is probably very low ( someone is probably going to correct the child anyway ) but you get to make $10 billion. Clearly the good it does to you far outweighs the bad it does to the child. You may not be evil in this case, and different people have different thresholds for it. There is no one so ethical that they would never lie to a child at any price. This premium increases with higher and higher crimes. Also the premium decreses the more desperate you are.
People with higher premiums for same level of desperation are better than others.
Stack ranking incentivizes people to stab each other in the back. Some people will take the incentive. It does not necessarily mean they are unethical.
It's definitely possible and it's happened to me more than I'd like to admit. I feel a lot of it stems from folks not stepping in and admitting theirs or others' faults.
I did a second tour a baml and did some of their release management. We hired a new guy who was technical and smart, but never listened to training since he was busy texting all the time. So, when he typo'd and all builds started failing, the fault was put on my shoulders and didn't want to admit fault. My manager wouldn't listen to what actually happened. I think I was out of there maybe two months later after they chose not to renew my contract (which thankfully force broke some golden cuffs). The guy who messed up is still there, I think. tl;dr baml sucks.
Another time at a medium sized company, there was a new ~really cool~ NOC being built that everyone wanted to check out. My boss took everyone up, but a locked door was in the way and I was given permission to open it with a CC. A week later, two other guys on the team did the same thing, got caught by security, and almost got fired by the CTO directly. My boss, at no point, stepped up and admitted he did the same. When talks of firing started, I made it very, very clear to him that if those two got fired, I'd make sure that both of us would be fired, too. Nobody was fired, so it might have actually worked, but it's incredibly shameful that of five others involve, nobody stepped in to add correction.
That same place also created artificial competition by only having one fulltime opening for the 6 contractors. Lots and lots of subtle backstabbing as a result. When they gave me the position over the other guys who'd been there for a lot longer, things became very oddly hostile.
I have more stories like this, and yes, it happens at big, large and medium companies, but it's so much easier to do it at bigger companies.
You can easily find teams of 5-10 in any company with no duds in, but the nature of stack ranking is that every year someone is going to have to take one for the team. If you don't know who the sacrificial goat is, it's probably you.
Yes, but you're not stack ranking in only teams of 10. You're stack-ranking 10 teams of ten. Out of 100 people, I can probably find 10% who are duds vs. their peers, but they might you might have teams with 2+ low performers.
You take a high performing team and rank its members. You take a low performing team and rank its members. When the numbers go to the next level up, the worst person in the good team and the worst person in the bad team look the same to the system - even if the worst person in the good team is better than the best person in the bad team! This is how the system works in practice. Good people get shafted because they sought to work with other good people.
So in a vacuum, yes people probably perform on a bell curve. But that bell curve is self-aware and will start sabotaging itself if you incentivize it to.
That's exactly the problem. You're assuming that there is a single metric of "best" that you can rank people on. Even in programming, there are a number of separate skills: algorithm design, systems architecture, debugging, etc. Is a programmer who can sit down with an unfamiliar codebase and be productive with it in a week's time "better" than a programmer who is great at designing scalable systems from scratch? You need a mix of technical skills, not just a team overloaded with whatever people and skills a particular manager finds interesting.
I think the biggest problem with your statement is that you're trying to identify the best people, rather than the best teams.
I'm just posing the question. We can assume that the HR people at Goldman Sachs are not idiots. There was no obviously better system or they and many other companies would have used it.
So lets say you identify the best team (itself a flawed concept - is sales more important than development, since they bring in the cash?) , then do you propose to treat everyone in the team the same? What about the new hire into the team who is struggling and should never have been a developer in the first place? What about the lazy stoner? What about the girl genius who reworked the problem areas over a weekend? Should they all get the same raise because they're on the same team?
you're spending an awful lot of time worrying about hypothetical people.
your argument would be more convincing if you started to explain your reasoning rather than just throwing out a list of cliches of "undesirables" in your opinion.
ok, lets keep playing the hypothetical persons game then.
the lazy stoner? sure he smokes up at home and comes in to the office late basically every day. he's also an outside the box thinker who pioneers the use of new technology on his team, and is an expert debugger who can view problems from angles his colleagues haven't considered yet and maybe never would have.
the new hire who's struggling? its not her technical ability thats lacking, she's just going through some shit in her personal life. her father is sick with cancer and she's splitting her time between her tiny New York apartment where she lives with 3 roommates and her parent's house in the suburbs where she helps her overwhelmed mother catch up on laundry and sweeping the floors at home.
the girl genius who reworked the problem over the weekend? well she's an arrogant workaholic and constantly talks down to her coworkers and drains the morale of the team. she spends a lot of time thinking about ways to make herself look better to the boss and eagerly takes full credit for projects she collaborated on with 5 other people, but she doesn't mention that at her quarterly review meetings.
If your point is that its hard to measure people's true performance, then yeah, sure.
If your point is that everyone should be paid the same because its too hard to ascertain actual performance, then you better find a gig on Cuba or some other functioning communist society.
You can apply selection when you form a new team for a new project. Then you select the best people for that project. Ranking scores are unidimensional, absolute evaluations of people. They can be abused by adding penalties based on frivolous criteria.
In China, Ali Baba and others have created a ranking for regular population, called the "social credit". If you make friends with someone who has a low rank score, your own score will be penalized (they want to control who people make friends with and socially isolate people by descoring them). Also, if you watch too many American movies instead of Chinese. These scoring systems (and even Google's Page Rank) pervert the normal behavior of people.
On the latter point, I think this will become more prevalent in the future, and has really interesting applications in displacing trust as a dependency in interpersonal interactions.
Once we have more accurate profiles on individuals, and the profiles are shared more widely, you could make much more informed decisions about who you want to share public spaces, interact, work, live, or even enter into relationships with.
By not rewarding based on "the best" (which implies comparison) and instead based on performance. Set realistic and definable targets for such rewards and follow through. Additionally, communicate these targets clearly. Allow for additional rewards for team performance.
I believe this maximizes the motivation you get from rewards both on an individual and team level creating the "best" you can get.
Although, realize, studies have shown rewards generally don't impact performance greatly.
A good team has people with a mix of skill sets. The "best" people depend on your choice of skill, but multiple skills are required to excel and it's almost certain no one person will be best at all.
Sometimes it's the gestalt of the team that works well; sometimes there's simply no easy way to isolate contribution.
The moment your boss essentialy said, 'Hey, I wasn't there to determine to fate of the rest of your career at this company,' you should have walked right out the door. Who cares if he apologizes? If a manager doesn't watch out for their employees, they're not a good manager.
Well, as much as I disliked the manager, I don't think there was much he could've done. He legit had to leave to go somewhere. My gripe is with baml, not necessarily him.
It sounds like they used to be much better. When I joined in 2010, they were working on a shift towards a new management style with metrics and the whole nine yards.
That management style also resulted in their backend risk management system using rot13 for authentication because the developers "didn't have enough time to implement something better." Being able to read, in plaintext, their production mortgage database's read/write password was pretty interesting. I avoid them like the plague now.
Wow, that's pretty bad. Metrics-driven management is one of those things that sounds like a smart thing to do but it ends up being counterproductive. It's really hard to measure the things that are important so we have to use proxies. Except optimizing a proxy is not the same thing as optimizing the thing you care about. And this is how we end up with production databases using rot13 for encryption :)
Back when I was in management, I used to collect concrete metrics like "you keep closing work items, but it's been three months since you checked in any code" or "you're employed as a tester but you only opened two bugs last year and they were both by design". Of course people could game it, and maybe they did, but it went a long way toward identifying the people who were just straight-up asleep at their desks.
They might as well have, honestly. I found it on accident when tying to link my group's database to the risk management system, since it was also their internal cluster computing system. It was more or less a giant python object store, including code. It had zero documentation and since I worked second, I had to figure it without any devs online. Eventually found the password store after looking through other groups' production code. Turns out that double clicking on an object showed its code and anything linked to it, so once you have the password function object, you get all linked password hashes. Double clicking the hashes then ran its linked function - rot13 - and printed it to the screen.
This was in 2011, so they should've known better! I've seen some pretty bad stuff, but that was pretty up there.
That's really interesting. I was at BAC at about the same time as you (based on other comments), and we still had a non-numerical performance system ("Meets Meets", I remember getting, but I'm not sure what the two categories were).
I think I was told about it with some secrecy in mind, actually. My manager there felt really bad about the situation and wanted to clear it up before any fallout happened, i think.
I'm not sure how your manager felt saying sorry was enough given that him leaving a meeting early had a direct negative impact on your salary and bonus.
I wasn't so mad at him as mad as I was other things, really. I left within three months of that, I think. That was just one problem of maybe 50. Even with that shitty experience, it was still worth it with career in mind.
I was supporting both of BAML's FX systems, including doing Sunday market open by myself for a couple hours. It was my first "serious" job and at the time I didn't know what common job practices were, including proper pay. As soon as I found out I could make more elsewhere, I left.. with a 40k pay increase.
Considering the number and size of trades going through the platform, it's incredibly insulting that they thought my pay was industry acceptable. Hell, a year after I quit (the rest of the team was gone at this point), they wanted me to offer me a job in NYC, making me less than my job as php programmer (without knowing php) in Chicago.
Yep, I'm in Chicago. I did second shift support for merrill lynch's FX system mostly, but dabbled in BoA's, too. Your name doesn't sound familiar, though!
edit: looked you up on twitter - your picture doesn't look familiar, either. Sorry! :)
I was over in Rates & Currencies on the BAML side, in 540 W Madison. (I know they have gotten rid of that building, and I live in New York now, but... I enjoy nostalgia!)
Yep, same building. Not that many folks from Chicago on this forum, so definitely a small world! :)
I worked closely with Alberto V. and Gary M. who you might've worked more closely with. Made some decent friends with some of the rates batch guys, so if you're feeling sufficiently stalkerish, talk to them. :)
In hindsight, I actually do recognize you from your pictures. Almost entirely through passing, but I'm certain we've at least been in the same elevator together. :p
This is good news. I used to work at Goldman IBD, and the rankings system was really dumb.
The problem was that you had to rate people on a 1-9 scale. As you might imagine, this rating was done via a web survey where you clicked a radio button under a corresponding integer, and under the numbers were verbal cues. The one under 9 was something like "One of the very best people I have ever worked with".
When you get your review, they give you your score for each assessed area, along with the mean score for your position/group (e.g., "associates in IBD").
I distinctly remember that most of the mean scores were between 8 and 9. One was as high as 8.7.
Clearly you had a lot of grade inflation that made the data points of dubious value. Whatever system they chose to replace it is probably better.
DISCLAIMER: I can't get over the paywall and some of the above points might be in the article.
The way the system would get described to me by the people I know who used to work at Goldman was that the tech guys would all treat the system seriously and attempt to rate people while the traders would all rate everyone at 9 for everything, unless they hated someone or someone had done disasterously in that area, in which case they would be ranked at 7. The programmers would always be looking for excuses to get a trader to rate them.
It was always told as a parable about how bad at real life game theory the academic, logical, game theoretic minded programmers were vs the traders.
To poll meaningful opinion in a highly incentivized environment like that, ratings would have to be strictly relative. Not "is person p1 any good at quality q1, q2, q3?", but "quality q1, q2, q3, which one of those is p1's strongest, second strongest, third?". A decision maker could then use that data to extrapolate from qualities he can judge (even if it's as stupid as "attention to personal appearance") to those he can't. People who score badly on a visible quality they are obviously good at must be pretty stellar at the invisible ones, those who score much better than expected in a visible quality, well, there might be a reason for that.
Still, that system would be sabotaged by favor based raters. They would quickly rank qualities by perceived importance (I know I would) and blindly rate the most important category highest/lowest depending on popularity.
Enter sudoku rules: make it a square of persons/qualities and require both rows and columns to be a complete distribution. The challenge of creating a valid matrix should then take care of any remaining game-theoretic influence, leaving only honest opinion and a fair amount of noise (which is a much lesser evil than deliberate manipulation).
It doesn't say how good anyone is at game theory. It might be a proxy for how social they are. The outcome is based on how "selfish" people are, or how much retribution they expect in future iterations. I don't see how this could possibly be used to make a deduction about the understanding of game theory (or practice).
but this is _exactly_ what game theory is about, the parent probably hinted at a failed unterstanding of the theory as shown by your comment ;)
what i mean is, people may think they understand the concept of game theory but are often unable to link that to its practical consequences, as in this case.
I really enjoyed "Jane Austen, Game Theorist" which argues that Jane Austen systematically examined Game Theory years before it emerged.
The author also argues that Jane Austen has analysed in more depth game theoretic 'cluelessness' than has been done since, and interestingly, many of her 'clueless' characters show signs of being unusually interested in numbers, maths, card games, etc.
That's actually surprising and I think it's a smart way for people to respond. Rather than trying to fight a losing battle and complaining about the problems of such a system, they seemed to realize the better course is to simply give everyone excellent ratings.
It also likely meant people were more productive as a result of not having to deal with getting bombed with a subjective score and other harsh annoyances, so in effect they were simply maximizing shareholder value.
On the other hand, Goldman likely does attract some of the best people, so everyone may just have been being honest.
Note that the "give everyone excellent ratings" strategy is explicitly codified in the standard documentation that is exhaustively and thoroughly studied by every analyst and intern before they ever come near a performance review.
Look up at the top of this comment page. Under the headline is a "web" link, which does a search of the article for you. In this case the very first one is to the WSJ article, and it's not paywalled. I'll also note that when I manually try the google search workaround it no longer works for me, but this did.
Well yeah, the same way that at some schools you might be in the top 10% of your class with a 3.95 GPA and only in the top 25% with a 3.85.
They can calculate where you stand on the score, but when the question of whether you're in the "top bucket" comes down to whether one or two people out of the 6-7 who are rating you give you either an 8 or a 9 on a 9-scale, that's a pretty fucked up rankings system.
The problem isn't rating scales, it's the assumption that employee ratings need to conform to a bell curve and that the bottom of the bell curve needs to go. If the outcome of your hiring and firing processes is not random, this should not be the case.
I once had a teammate get an apology from our manager because he had been required to give a poor rating to someone.
Even if your hiring system is perfect, you'd still have a bell shaped performance curve. But since people understand that their hiring isn't perfect, they look for ways to keep improving. I wonder how many of those here who get their panties in a bunch over rating on a curve have actually had to manage, and make a profit from, largish groups of people.
"The normal distribution is useful because of the central limit theorem. In its most general form, under some conditions (which include finite variance), it states that averages of random variables independently drawn from independent distributions converge in distribution to the normal, that is, become normally distributed when the number of random variables is sufficiently large."
This is why engineers get frustrated with managers that pretend to know math. Employees are not selected randomly and companies like to apply curves to even medium and small departments. A normal distribution does not apply.
Well, if your hiring system is perfect (and your talent management, and your leadership, and your product, and luck is on your side) you could have a giant spike on the right: all perfect people.
That's not realistic, so there's some sort of distribution. It's probably at least somewhat bell-shaped. Even when the average is very high, and your worst performer is still "pretty good", they're still bringing down the quality of your staff. If your process is that good, you could replace that person and go from good to great.
This is all hyperbole: this only applies to large populations, this assumes you can normalize (hard), and this assumes no one is lagging (never true).
My own megacorp (which is mentioned in the article) when to a warm & fuzzy review system some months ago, I don't really like it. They used to do stack ranking behind closed doors (although they vehemently denied it) ... which only bothered me because they didn't tell us where we fell in the stack. Let me know where I stand compared to everyone else in my bracket, that's all I really want.
FWIW: the Goldman recruiters at my school perked up when I said I was in computer science. They went over the "fun" stuff that their interns got to do (baseball games, travel, etc) rather than in any of the problems we would be working on. Seemed off-putting and made me question what kind of work environment it would be (who wants to develop in a suit?).
I ended cancelling the interview with Goldman and going with another company in the finance sector that has a long (decades) reputation of being genuinely great to employees.
Well, I can't read the article because I've chosen to focus my newspaper subscriptions on other, better, eh, 'peach-colored' newspapers.. but I worked in investment banking in IT for a number of years. Pretty much almost 10.
I've been thinking about stack ranking for a long time. In one place I worked, a large rival of Goldman, you would be ranked (secretly) on a curve, typically from 1-5. If you were the manager of a group of, say, 6 people, you couldn't just give everyone a 5 or a 1. You had to even it out, pepper in a few different numbers in that range. Maybe you had one 4 or one 5, say. This would go up to your manager, who would, in addition to rating you with your peers, would take a look at what you rates your employees and 'adjust', ensuring that the curve is adhered to, maybe moving numbers up or down depending on how they felt about your employees and also as compared to other groups (since it's a hierachy, a pyramid where each layer up managers more groupings of people).
This system was applied to the layoff selection (1's are out, some 2's are downgraded to 1's as the rankings made their way up the chain of command), and may have applied to bonus system as well (which is more complex). There were layoffs at least yearly, most years. Goldman is worse because the reputation was to always cull the bottom 5%, and I heard 10%, of staff yearly and (in good economic times) get a new fresh set of people not long after that.
Regardless, the takeaways of a system like this is that, regardless of your INDIVIDUAL performance, what matters is your performance relative to your peers. There are few issues with this.
The first issue with this is that the ranking system like above, or any ranking system, makes a lot more sense when you're in a role that has tangibly quantifiable measurements. For example, sales, or -- especially, in a place like GS -- bond or equity or other trading. In either one, if I make the company 1M, and you make it 1.2M, it's clear you added more value.
Well, what about IT staff? Is it lines of code? No, of course not. So there are these nebulous, subjective measurements pretending to be something more useful than they are. Performance reviews, the impact/importance of the projects you completed (which you didn't even get to CHOOSE, in most cases), and (at the place I worked) items like letters from senior managers about you that were positive, all counted.
The second issue is that competition doesn't necessarily make the work done better or more efficiently. The worst part for me was competing with people on my team. Instead of the team competing with other teams (if we want to ideologically just adopt the notion that competition makes everything more efficient and effective [which I think is overstated heavily], at least team competitions are a little better), we would compete with one another.
There are two ways to compete. I work harder, or better, or perform better than you. The second way is, we perform the same, or I perform the same, but I undercut you somehow. What's an example of undercutting? Well, let's say that on conference calls when it was your turn to speak I'd take knowledge about your project and use that to show that I know more publicly, or let's say that you undercut me by taking over my project and that of others, to show the boss how smart you are, etc. Most managers are so busy themselves that they don't have time to notice these things, which seem like petty infighting / childishness in their eyes. I saw the 'project takeover' scenario happen everywhere I've worked in these competitive environments. I saw one guy take over a bunch of people's projects, find himself overworked, complained, and got himself staff to work under him.
Hey, it's not easy to figure out what to do with regards to ranking, or whether to do it at all. When you're a full-time IT worker or professional in America, you get a salary, but it kind of doesn't say your hours in your contract; or if it does and is (rarely) enforced, at the end of the day in a competitive environment it's not fair if some people have to pull the weight of others. So there has to be some differentiator. On the other hand, what I've seen in IT is a downward spiral of misery. There's always someone on the team who puts in 60 hours a week so he can show how smart he/she is, trying to get that bonus or promotion above you.
And it's entirely possible that the above is more pronounced in the large, Northeastern American city (and industry) I live and work in.
stack ranking works in certain situations, doesn't work in others. If you pay way above average and don't have that many employees (ie, your needs relative to the population are low) you can usually get away with it. So, it works for the netflixes of the world. Amazon has moved away from it somewhat as they realizzed that they were running out of engineers to spam to hire.
I haaate these numerical ranking systems. Bank of America had one when I worked there. Managers would have meetings with each other to match their workers with ranks. Really, really long meetings. So long that my boss apologized to me after the meeting to discuss my ranking.. because heleft before my name came up and there was no followup meeting. I ended up being ranked towards the bottom, I didn't get a raise, and the bonus was meh.
When I left, they told me that my performance was great and that they wanted to give me a substantial raise, since apparently, my pay was far below what it should be. They then had to replace my weekend shift with four others for a while. Most of the team was gone a bit later.