> To put it bluntly, I’m not sure I trusted Infosys to revoke this key in a timely manner. So I did it for them with aws iam delete-access-key --access-key-id=$AWS_ACCESS_KEY_ID, and now the key is useless:
Hilarious. Infosys is a known "mass recruiter" in indian colleges. WITCH (Wipro, Infosys, TCS, Cognizant, HCL) companies is where talent goes to die. No competent employee stays in those companies (from what I've witnessed). Wouldn't be surprised if this turns out to be just the tip of the iceberg, because putting people with 6-12 months of programming / computer "experience" (that they only signed up for because of the money) in charge of major production systems is a recipe for disaster.
I had some contact with Wipro. It was their standard operating procedure to call us up and yell at support team members that X "Hasn't worked for months and you haven't done anything." + escalate up the chain as high as possible to put pressure on the tech support staff from some other vendor, when in fact they just opened the ticket. They would lie and reference the first old ticket they could think of and say it was the same issue (it never was, they wouldn't even lie well enough to reference the same equipment).
They would declare everything was a P1 ticket and demand it be fixed immediately. Then we would get some output from the machine or even remotely access it and find that outside of testing at the factory this was the first time it was powered on. When we would ask them for configurations ... they were evasive.
If you got their end customer on the line you would find that they had been lying to them for months. This happened a lot ...
We have a FTE who came from infosys and he's very good. I have such a hard time squaring that with the team that submits an initial PR with the bin and obj directories checked in, then follows it up by adding .gitignore.txt file before FINALLY submitting a .gitignore file. And then finding them representing currency as float, or finding catch statements with a single line that rethrows it, as below (C#)
// this form throws away the stack trace from the original exception.
catch(Exception ex) {
throw ex;
}
And when asked why this exists they add logging to it
I could go on, but the ole eyebrow just twitches whenever I think about infosys.
But then I see this other person who came from infosys. It's like trying to understand how that 6'11" basketball player came from that family who has no one over 5' tall.
Rationally I know strong technical folks can come from these companies, but damn... how? There's another poster claiming everyone makes mistakes, but no, many of the mistakes they make are not reasonable.
He must have joined Infosys at 'on-campus' recruitment. Infosys and other companies might have visited his campus, he gave a sort of test and he was selected. Sometimes, once you pass an interview, you aren't allowed to try for other on-campus companies. So he was 'stuck' with Infosys.
The other reason is that programming as a hobby during college isn't a thing in India. (This might have changed in recent years). So you only get a chance to really mature as a programmer in the first few years out of college. So when he was ready to move on from Infosys, he had matured, but still had the Infosys 'stigma'.
And then it's really a numbers game. Infosys has hired millions in the past decade or two.
Programming/anything to do with PCs in India is a rich man’s hobby. Tinkering is not encouraged in colleges in the country. In any case this is a big country with a lot of talented devs. But for the same reason the pool of mediocre programmers is also pretty big.
>Programming/anything to do with PCs in India is a rich man’s hobby.
This is not true. We had a good culture of self assembled PC enthusiasts in our district. This was in early 2000s. In fact it has reduced now maybe due to lack of interest or something even though the prices of PCs have dropped significantly. I see many people use their PCs as locked up phones with no curiosity of hardware.
>Tinkering is not encouraged in colleges in the country.
This sad state is still present to this day and it has become worse as the hardware gets more and more locked down. I don't see much interest in Linux in the younger generations.
It is absolutely true. Just because you don't consider yourself rich doesn't mean that these devices aren't out of the reach of a vast majority of the population.
Almost everyone has access to a smartphone, but most of those are poorly made, overheating pieces of plastic - barely suitable for use as a phone, let alone as a computing device on which you can learn something.
In my second statement, I'm also trying to say that though you have a huge number of people in the workforce who haven't had easy access to computing devices, there are also many who do (in absolute numbers but not in percentage). But the numbers are such that given an average Indian programmer, they are more likely to be someone who got educated in a substandard setting among unmotivated peers who have an aversion or even a fear of tinkering and putting a big investment (like a PC) at risk.
People who have money to get a PC as a hobbyist do it for unworthy reasons - like the chance to play garbage tier games like pubg or whatever, or becoming an influencer. This is a generalisation but it will affect the probabilities.
The resources that we have now is way better than what we had in our time. The interest amongst students have been more or less the same.
>that given an average Indian programmer, they are more likely to be someone who got educated in a substandard setting among unmotivated peers
This too has not changed much from our time to now. The changes I have noticed in students is the rise of memorizing leetcode type problems due to the plentiful jobs now which need this skill for interviewing.
Tier 1 companies generally hire from tier 1 colleges. Tier 2 and tier 3 college students are either ignored or not able to make it due to lack of quality education.
But those talented students take up any job offer they get (I.e. WITCH). when they get experience, they switch to higher tier companies.
There are many talented folks at WITCH companies, they just don't stay there.
Yes. There are a number of highly skilled and talented people in Infosys and the other WITCH companies, but they're generally staffed only on the most prestigious projects and tend to move on fairly quickly. As another commenter said, in most cases they got recruited straight out of college.
So it's not that everyone at companies like Infosys are bad, it's that their hiring standards are so lax and hiring rate so high that the large proportion of their engineering people are mediocre at best, and that's why most engineers at European or American companies would've been exposed to.
The typical model of a WITCH engagement is to get a new client project that requires, say 100 engineers, and immediately go to market to hire 90% of them because they don't have a bench. Screening is minimal. They're then heavily micromanaged on the project for the first few months, where it's expected that at least half of those people will fail and either their manager or the client will demand they get rotated off and then sacked. They're replaced by another cohort freshly hired and the process repeats until you have a stable-ish team of good-enough competence about 8 months in.
It works because it's still cheaper and easier for big corps and big projects and the delivered quality is fairly shit, but still acceptable. And the margins are so good that in the rare event there are late delivery penalties they're fairly easily absorbed.
Emphasis can be a good way to get visa endorsement for finding a better job. Also sometimes people who are booksmart are not streetsmart and end up working for a bad company.
Fun fact: Mozilla projects are now developed in part by Cognizant Softvision, including Firefox for Android. Their employees are everywhere on Mozilla bug trackers, and their numbers seem to have increased since 2020, right after Mozilla fired a quarter of its workforce.
Luckily, if https://wiki.mozilla.org/QA_SoftVision_Team is any indication, they're mostly babysitting test suites and reviewing submissions to the add-on store and stuff. (That and developing Firefox for Android, but Firefox for Android has sucked and included tracking SDKs for years now.)
> and are on the path to go broke. Sadly, they’re the only mainstream competition to Chrome.
Is it overly cynical of me to wonder if this is Google's doing? Setting someone to infiltrate Mozilla's management and sabotage it, with the long-term goal of killing all serious non-chromium alternatives.
I don't know about the woke thing, I figure it's more likely to be about removing ad-block friendly API's in Manifest V3 (and presumably even more hostile changes in some future Manifest V4).
Nobody wants firefox to stay around more than google. Its one of their few escape cards in an antitrust trial for browser monopoly.
Google wants Firefox to stay around, but in a form that's much closer to Chrome. Of all the browsers that still have significant userbase remaining, Firefox is an "anomaly" in that it's one the user has more control over, and Google is slowly trying to change that.
Already happened years ago, if you want the old Firefox with XUL extension support, full themes and zero tracking as it was before they started copying Chrome in 2011, give Pale Moon a try. It's an independent fork, the last independent browser left, with its own rendering engine Goanna that is a fork of Firefox's Gecko.
Imagine being a top performer doing great work for a company whose managers insist on wasting your time putting you into needless meetings getting you to explain how you're doing everything all through badly communicated text with typos and misspellings.
Someone hired those clowns as contractors as extra in a previous job, to loud protests from our development team. They produced what was quite possibly the most chaotic, copy-paste, typo-laden code I have ever seen in my life.
I've seen Infosys-produced code that there was no way it was going to work... turns out that after I googled it, multiple lines were straight 1:1 copy pasta from multiple StackOverflow answers - just jammed together in the hope that something would work. I was shook.
This is unfortunately common even outside of Infosys. I've experienced it at several of my former employers, although admittedly more in China than in other countries I worked.
It's interesting when you sit beside a developer who does this kind of stuff in a pair-programming context, because it immediately becomes clear that they really don't have a clue how to read and understand code in the abstract. Their process is literally copying and pasting stuff that seems similar and then running it until some arbitrary happy path test passes, not considering that it might only be passing by accident, or that they might not even be testing a real business scenario, or that there are now a bunch of unused and misleadingly-named variables floating around. And when you point that out, there isn't even a lightbulb going on that perhaps they should try to clean things up or adapt the pattern to better fit the specific use case.
I've always attributed it to a mindset that doesn't really take quality into account. And it's hard for me to argue the point when I have also been "guilty" of doing a quick hack solution or employing YAGNI to build something that might not be DRY or especially elegant but does work to solve the problem. People who just throw everything at the wall until something randomly sticks believe they're doing the same thing. Who cares if the code is unmaintainable or not performant? Who cares if there's a bug? They still get paid anyway, and the corporate machine just keeps rolling on. So - from their point of view - why make the extra effort? For me I think it's just a neatness or tidiness compulsion that makes me want to try to make code clear, robust, backward compatible and maintainable. But realistically even if I didn't do that, I'd probably still be 20 years into my career and working as a senior dev, so what's the difference?
Somehow this triggers memories of (among all things) taking exams when I was a student.
Unless you prepared well, there's often some exam questions you are clueless about, and yet there's usually no penalty for writing some bullshit in the hopes of accidentally getting a partial score. So what students are trained to do is to write whatever bullshit that seems to be relevant and hope for the best.
I realized the mindset that makes me a quality-conscious programmer is actually the anti-thesis of this. In fact during my later years I almost couldn't do that exam-bullshitting any more. It feels so bad writing something I don't understand that I almost couldn't do it.
This might be offtopic, but I guess many people who don't have a natural OCD-tendency to deeply understand their work and care about tidiness might have to actively unlearn what they trained for at least a decade in school...
I don’t know you at all aside from this post, so I could be off the mark, but I suspect you need to find a better place of employment. Places where people exercise discretion and aim for quality and ship quality do exist. They are not the norm, but they absolutely exist.
Thanks for the suggestion. I am pretty happy in my current place of employment where I fortunately haven't (yet) encountered this, but I have definitely been surprised in previous jobs where I encountered it despite hiring standards that theoretically should have rejected this type of candidate at the outset. I have even worked with ex-FAANG colleagues who had this approach to development. I have come to think there is only so much you can do to try solve this in the hiring funnel... Eventually people who don't code very well or don't take code quality seriously will slip through, especially in larger companies.
Man, I echo the same sentiments. Even I too have this compulsion for neatness and tidiness and find it near impossible to work on completely messed up code.
>But realistically even if I didn't do that, I'd probably still be 20 years into my career and working as a senior dev, so what's the difference?
Similar thoughts. Looking back I think about all the time I had sacrificed to make myself better but for what? There is no value for this as I have seen totally incompetent people still standing and moving much ahead. The corporate juggernaut does not give a damn about workmanship or quality, something I learnt later in my life as I took a pause to catch my breath.
These companies literally do not want to hire competent people, because they know they won't stick around. Their business model is to hire the absolute bottom of the barrel engineers, pay them the tech equivalent of minimum wage, and sell "consulting services" to overseas firms that don't know any better.
I have had some ridiculous situations with them. For one the developers in Bangalore Delivery Center 8, and what do I know, all of them?, had to work with their private computers.
A unix engineer could only work with Aix Tar and would not touch GNU Tar on Linux, because his manager had not approved it.
Onshore engineers flying home to India due to a stomach ache, instead of seeing a doctor for free in the host country due to being afraid. Of course messing up the flow of our projects.
10/10 will leave jobs to avoid such projects and situations again.
> Onshore engineers flying home to India due to a stomach ache, instead of seeing a doctor for free in the host country due to being afraid.
This feels like an exaggeration to me, although I'm open to hearing specifics to the contrary. I know of (non resident) immigrants who delay medical visits and treatments until they get back to their home country, but flying home (spending a good deal of money on air fare) for just a stomach ache sounds pound foolish, which immigrants generally aren't.
> Infosys is a known "mass recruiter" in indian colleges. WITCH (Wipro, Infosys, TCS, Cognizant, HCL) companies is where talent goes to die.
This could be true but you cant really generalize and it has nothing to do with the article. Infosys is not the only company leaking keys online. pretty sure tons of Amarican companies have done that
This thread is full of generalized insults at a million people based on where they work. If someone did the same based on a different attribute of a population, they'd be banned.
I've worked at one of these companies but left over a decade ago. I know how we're looked at when we do client work (part of why I left). Some of my colleagues were less competent, true. But, some will wipe the floor with the client employees we did the work for.
To WITCH employees:
If you are an employee at one of these companies, remember you are not the worst. Many of you come from humble backgrounds and are just learning the ropes. The world is cruel. It is a tough place, and you will be discriminated against. This is your fuel. You've already made great strides; keep going. You have to.
I'm not quite reading it that way, but if that's the case I completely agree: You should never insult people based on where they work, or for any reason really. What I see is a general criticism of the companies, their culture and business practices.
InfoSys is not a company I worked with, so I can't and won't comment on them. TCS is a company I have had the misfortune to encounter. The problems with TCS is numerous, a few examples: they oversell, you're denied access to consultants that can actually help and they will always prefer to prolong an issue, rather than escalating to senior consultants. There's no incentive for one of their consultants to be pro-active or take responsibility. There are so many departments/team and layers in their organisation that there's always some one else to point the finger at.
The consultants are TCS aren't stupid or incompetent, but they also aren't being helped, pushed or motivated by seniors or their management. I do got the feeling that they would be reprimanded if they where to escalate an issue. In a meeting with TCS I suggested added 8GB of memory to a VM, as either a temporary fix, or a sort of "let's see what that does for the client". That suggestion was rejected because: It wouldn't fix the underlying issue (which was true, but they also didn't want to upgrade Java or the operating system, which was part of the problem. The OS being an old unsupported version of CentOS), and also wasn't something you could "just do". That would require involvement from 5 or 6 other departments. A month later, someone finally caved in an escalated to a higher up TCS consultant, which just added the memory as a fix until the service could be migrated to a new OS and JRE.
Anyway my point is: No, it's not the staff, not as such. They skills are for the most part perfectly fine. The company did have true experts available, if required. It's just that the culture is a really bad fit for western style companies, if you're in Northern Europe it's an even worse, because we don't share many of their values and fears. This could be solved if the Indian companies better understood the market they're selling into, because they do have the technical skills. As it stands, people like me get annoyed that we have to tell the clients that we can't fix their issues, because someone in Mumbai is afraid of looking bad to their boss or ask a colleague for help. If it has to be like that, then at least have the balls to tell the client yourself why you don't care that their systems haven't been running right for a month.
That is a very compassionate comment.Thank you!. It is such a sad thing to make sweeping generalizations, I know of many ex employees of these organizations in FAANG, startups. Agreed that the ratio of great technical talent may be small, these companies have 300k employees, a vast majority is maintaining a legacy application that is keeping a business alive somewhere or processing someone's health insurance claim or something important thereof. You will find some really smart people doing products like Finacle, a well adopted core-banking software.
What was done was bad and than talking of bad practices (so many exposed buckets in AWS, miners using compromised EC2 instances from github repos) a vast majority of the discussion seems to be sweeping generalizations of how every single person employed in these companies are!
There is something to be said about the repeated displays of incompetence though. My own experiences working with WITCH employees have mirrored those of the other comments and that of the article. It is not wrong to criticize the methods that they pursue, nor the fact that they do not wish to learn from their mistakes.
Most companies the size of WITCH do not utilize access keys nor add them to source control. While a developer may make a mistake, you would expect there would be guardrails around the development process, either by way of an automated scanner or a more experienced software engineer catching it as part of a code review. The fact that none of this happened is quite concerning, IMO.
You could also perhaps say this is a management problem than an employee problem; and while that is true, such distinctions are rarely made. As an example, I'm sure you've had bad experiences with customer support which you simply summarized as "The support rep at Corp X sucks" when talking to other people; whereas the truth might be somewhere closer to "The support rep was out of luck because they didn't have a process to do A, B and C because management didn't think of it."
> Most companies the size of WITCH do not utilize access keys nor add them to source control.
Most companies the size of WITCH do not use barely out of college engineers for rock bottom prices, driving them to deliver, features, features, features at all costs.
Literally all costs. It's a lot simpler to work with AWS if you can just plonk your full access key down everywhere, and even someone just out college can understand it.
Conversely, dealing with AWS Roles/Profiles and permission is a whole separate profession by this point.
It is a tough place, and you will be discriminated against.
Calling out incompetency which exposed privileged patient data is not discrimination.
Rough analogy: you don’t want a pilot who flunked basic aviation class to fly your plane and it’s not discrimination to keep him or her out of the cockpit.
I am from India and live here. I usually find it offensive the way HN becomes racist, subtly and directly, when topics around this subcontinent, especially India, come up. This is anything but that.
I think people are unnecessarily being much more considerate and respectful than this company and its people (including the British PM’s father-in-law) deserve.
Infosys and anything or anybody related to it are worst of the worst.
I’ll take a shot at being brutally honest. I feel this in a different way. I grew up in a conservative family with some racism in its more distant ranks, thankfully with a more liberal mom to balance it out. I grew up fairly well off in a white area, where there were only ever two families of color. Both families moved away in a much shorter time than the average.
I’ve noticed that for awhile I had carried an innate aversion to offshore outsourcing, but only when it’s predominately non-white. It’s difficult to rid yourself of these intentional or unintentional exposure based thought patterns.
I had the privilege and good luck of ending up in a position where I ran an educational, science focused nonprofit. Then I started a business that had needed skills far more expensive in the US, before we could quite reach that level of expenditure. You learn quickly in those kinds of situations that if you carry those innate perspectives you can end up locking yourself away from some excellent talent; capable people who can work magic if you set them up for success.
This comment is only in reply to the topic of race. I’m not making any judgements or assertions about Infosys or any company in particular. Some companies and some people are bad at what they do, and that’s a global truth that is blind to race, culture, creed, politics, and anything else. I’m in full agreement that this type of security failing can, will, and has affected any company no matter what their employees look like or where they are based/operate.
I don't see this as a India problem, it's really an incentive problem.
In my experience the further you get from the money, the less of a shit people give. At a 5 person start up the result of any effort you put in is considerably more noticable, you don't have to share the credit of a innovation with a thicket of business analysts, scrum masters, executive vice presidents, etc. In that type of environment people tend to put more effort in as generally a sizeable portion of the rewards for that effort will find it's way to them. (Side note: this has changed with the innovation of Hollywood accounting[0] for start ups, and the number of truly innovative start ups has also seemingly declined)
Now think of a large company. The rewards tend to be nearly entirely rank based. You are a Software Engineer III, that pays between $x and $y, if you want a promotion you'll need to change fields into management. Perhaps a really bright idea or large effort will result in a small bonus, so you still have some reason to put effort in but probably won't go crazy.
Now go one step further, you are a employee of a 3rd party firm working for a large corporation. A big part of the firm's value prop is that they are cheap, as in they demand less of the reward for effort, they share a small portion of that with you but also have their own thicket of business analysts, scrum masters... you get the point. At that point honestly why bother? You have so many middle men between you and the results of your efforts that it's very unlikely that you'll ever see any meaningful reward. Just do what it takes to not get fired.
Thank you for this! I've said the same thing and had to deal with salty downvoters earlier today. Companies pay WITCH companies billions of dollars for their services yet a lot of pretentious hackers just don't see the value.
That's just based on the ability to convince the management types though, and I'm sure you've heard of the phrase "nobody got fired for buying IBM."
For an executive, it's easier to justify outsourcing to a large consulting firm simply because of the security afforded by the choice and the ease of justification; rather than any technical abilities they may or may not possess, and certainly it does not imply its correctness.
The anecdotes you hear are from a engineering perspective, which is where the consulting firm has to walk the walk, exposing their true abilities. It is incorrect to dismiss that as being "salty" or "pretentious", and tint them with an angle of "discrimination". The lack of processes and guardrails in these consulting companies is an objective fact.
Steve, how dumb do you think management is? Why would a company spend >=$1B+ in OpEx and CapEx just because somebody convinced them instead of seeing any technical value whatsoever?
Did you ever have an interaction with your manager where they could not understand the technical details of a project and insisted on only a high-level overview?
People understand the world through dimensionality reduction and lossy compression of information in domains that they are not involved on a daily basis. This causes an inherent issue when you stack these phenomena across multiple organizational levels; this is how you end up with Intel's or Ballmer-era Microsoft's management failures, to use some non-WITCH examples, or the issues that we're talking about in this article.
Management is human, just like everyone else. They absolutely make their share of truly stupid decisions. It's just they have more power, so their stupid decisions cost way more and affect more people.
There is nobody responsibly for hiring of new people into any of the bannable groups, or firing from them. There is management at Infosys that is 100% responsible for the apathetic, rot that engulfs it.
The funny part is WITCH is propped up by these very companies complaining because they themselves are least bothered about quality. I have seen many executives in suit and tie visit them, get treated like royalty and shake their heads and talk bullshit.
I always used to wonder how can someone be so stupid repeatedly but then I learnt along the way that engineer's opinions hold very little value in the way of making money at the lowest cost and quality possible that they can get by.
> based on where they work. If someone did the same based on a different attribute of a population, they'd be banned.
Why is that surprising? Are you making reference to judging people on the color of the skin versus where they CHOOSE to work? I'm don't know anything about WITCH companies, but this is a serious false-equivalence.
They lifted millions of us at least a class or more, financially.
Lower -> middle
Middle -> upper middle
Some even got rich.
In a caste discriminating society, they leveled the playing field.
Their business partners continue to do business with them. I remember an internal story, during the GFC, we worked on credit for a client who couldn’t pay their invoice($ millions). These companies are not angels, nor they only hire the best. But they’ve been the launchpad for millions of IT careers that wouldn’t have happened otherwise.
Getting a job in India is not a joke. And that's saying something.
Most poor people in India, are not the same poor you see in the USA. Many people who make it to WITCH companies are likely succeeding despite all odds, and
are starting their career at such companies, while they can get trained, and work on projects and later use the experience to do some thing good on the longer run. Several lucky also get overseas travel opportunities many even settle outside India.
Sure things are way less than perfect. And if you come from a rich family do not join a WITCH. You can either wait out for a better job, or may be go overseas for studies or just try to immigrate to some western lands.
Most people complaining about WITCH companies are typically from a background which is already better off. And they generally find such companies to be downgrade from their current social class. The remainder do just fine.
I applaud any bad press on InfoSys.
I picked up contract gig through them a few years ago. Here are some of the takeaways from my short lived experience:
- It took them over two weeks to send me a computer.
- They cancelled PTO for everyone. (this was the most egregious single thing they did)
- They had absolute worst internal site for accessing HR documents and accessing personal resources. Just a maze of links. You could only access it via Internet Explorer (I swear I'm not joking). Everything took forever to load. It was like stepping back into 1997.
- When I gave my 2 week notice, they refused and said I 'owed' them at least a month. LOL not sure how they think they can control people like that. I gleefully told them to 'deal with it'. This happened about 45 days after I started as it became obvious very quickly how bad this company treats people.
was in a client position (Infosys was contracting for the companied I worked for). Absolute worst processes in the world. At one point they blocked legit dev domains in their firewall and took 3 weeks to unblock a mongo db after vehement protests. DON'T touch Infosys with a 100ft pole
> When I gave my 2 week notice, they refused and said I 'owed' them at least a month. LOL not sure how they think they can control people like that
To be fair, in many countries (probably most developed ones) there are regulated mandatory min and max notice periods. E.g. in France the standard is 1 to 3 months, negotiable of course.
Wow. Really crazy. I know it was not right to revoke the key, he touched into their system. He probably broke someone’s production.
But it was also absolutely the right thing to do. A god mode key floating around for over a year unrevoked, with real human beings’s medical data on the other side… I am glad the post author revoked the key. It is probably too little too late but they did close that door and maybe saved someone some pain: not the negligent development team, but a real patient and human being, perhaps many of them.
I tried to highlight this in the post, but the key is a personal user one tied to an email, and the worst that I expect would happen would be that some training scripts break.
If this was a production key or something that seemed like it would cause financial harm/downtime, I would have never deleted it.
Honestly, with this level of competence I wouldn't be surprised if the same admin user credentials were used in application/lambda processor/whatever there is. Not at all saying you shouldn't have done it though!
I’ve worked on a team where Github was the one who reached out about a leaked AWS secret key, not AWS. They apparently usually do this a few minutes before the key makes it into their search index. It’s not much but it’s better than nothing.
You might be horrified by how many shitty developers want all the good guardrails GHE provides switched off, and how many managers will support them because they're a "superstar who gets things done".
Can't help but be annoyed by the flock of pretentious hackers painting every Infosys/TCS employee with a broad brush. One might say this particular leak is bad on part of Infosys and they must be held accountable for this. But calling the entire company incompetent is just lazy and stupid.
They make more than $3B in free cash flow, they are worth more than $80B in market-cap and they gainfully employ more than 100k people. Folks commenting here about the "competency" of a company should realize this. Most of their clients are based in US and UK. These companies have been using Infosys' services for decades and also have locked in deals for the coming decade. If a company was really that incompetent, it really wouldn't be on the scale they are today.
You might call them a "boring services company" but they matter a great deal to a lot of people. Less pretension, more focus on "value", please? :)
I think you are misunderstanding what these companies have deals with Infosys for.
It's not because they're so competent, it's because they're a convenient scapegoat when things inevitably go wrong.
Things inevitably go wrong for them because people hiring a company like Infosys do not want to be told how to do tech by competent engineers (and are probably not able to distinguish competent from incompetent engineers in the first place).
Yes -- among other reasons, none of them related to quality. WITCH companies are notorious for this. Accenture, for example, boasted similarly ginormous numbers for a contract with the NHS (11 bn GBP), which never produced any working software before being scrapped after several years.
Absolutely. Do you think any manager at Daimler wants to say/justify “I went with this noname 10 person company in the midwestern US” over “I went with Infosys because everyone in the G500 does, and have you seen those prices? I have 100 people working on this project where otherwise I’d have only 10.”
And what’s more, if one of them realizes their mistake, do you think any of them want to admit that to themselves, much less their boss, after sinking billions into it?
If the focus was primarily on value, a lot of comments would be significantly more scathing in significantly more cases. The fact of the matter is that if you work for a company that produces trash, that is fine - everyone has to eat. But nobody owes you respect for it.
Eh? First of all, I don't work for them. Secondly, what makes you think this company produces trash? Vanguard recently signed a $1B+ deal with Infosys to help them with cloud migration and other services. Why the heck would an established client like Vanguard pay a such huge amount for no reason? You are either ignorant or just don't understand the business value companies like Infosys provide. I'm guessing you are a Software Engineer?
> Why the heck would an established client like Vanguard pay a such huge amount for no reason?
For the same reason the Canadian government spent billions on IBM, and Hertz on Accenture, with a complete dumpster fire for a result, and other organisations still trust Accenture and IBM (Kyndryl now) with their money. It has never been about quality with these types of contracts.
I don't understand why you are being down voted. I disagree with you that they provide quality though. They don't. It is also the case that the company that hires them provide any quality. All are in it for making money with lowest spend and quality that they can get by. Very few obsess over quality and ones that do are vertically integrated to control quality in each step of the process. Very few American companies are like this.
The engineers who complain here don't have any influence in the decision making or otherwise they wouldn't be crying and complaining here.
The engineers here need to understand the world doesn't function well if everyone waited for 100% quality before shipping something. Business trumps Engineering always. And I say this as an engineer myself.
The numbers don't matter, because they're not about the core issue at all. My guess is that the post reads as if you don't understand that money can in fact be spent wrong and you are downvoted for this reason.
I really wish this surprised me. The number of people who completely understand the stack they are working on is shrinking, even as the size of the stack grows.
The power of computing is such that every organization on the planet is forced to lower the bar to get people who are marginally competent, even if they lack attention detail and cannot be relied on to solve problems of this sort. This kind of leak is the result.
I don’t think there are any people who understand the full stack. I don’t think anyone like that has existed in computing in a very long time.
It’s truly impossible for a single human to actually understand the physics of electronics, the world of CPU micro-architecture, packet shuffling network equipment, the nuance of CSS, and the never ending complexity of UI/UX design.
The only way this statement could be accurate is if you arbitrarily start cutting parts of the “stack” out.
I disagree, it takes lots of time but it is possible.
Personal example: I have an electronics engineering degree that was 1 semester short of a physics degree, so I learned quantum mechanics, electromagnetic field theory, transistors, and how to create a CPU (I even created a CPU out of simple gates and way too much wire wrapping). I love computer software, so I learned assembly, how to write compilers and operating systems, and libraries. I have configured network hardware and written network software at various levels. I've also used CSS and implemented UI/UX. I've written code in many programming languages, including JavaScript, Python, C, C++, Java, Ruby, Rust, Common Lisp, and Scheme. I eventually got a Computer Science degree as well.
None of these things are magic, and the info is relatively easy to get. You simply have to keep learning and be willing to try new things. It can be fun, too.
Yes, today it can be helpful to specialize at any particular time in your life. But I think it's best to use that as a launching pad to branch out.
It's relatively easy to learn a new language (for example) when you think, "this is like x, this is like Y, etc."
I guess my key point is that you always need to keep learning, and don't box yourself in too much. Ideas from elsewhere will show up... being aware of them makes it easier to use them and be ready for them.
If "full stack" means electronics up to JS, then there are probably quite a few people who can work at all those levels. Although a minority, at least they can understand a "fuller" stack than most, unfortunately.
I dispute this: I do not think you need to understand the whole stack to know using what effectively is "god mode" access is bad practice.
Even if I pretend I don't know anything about AWS, if somebody handed me credentials with access called "FullAdminAccess" and told me to use them for my little script that only needs read-only access to S3 I would be extremely skeptical.
The reality is that the culture at Infosys seems to place zero value on security of customer data.
>Even if I pretend I don't know anything about AWS, if somebody handed me credentials with access called "FullAdminAccess" and told me to use them for my little script that only needs read-only access to S3 I would be extremely skeptical.
If you ask for an access key for your little script and get one, you usually only check if it works for your case and not always check if it has any other access, so I can easily see it happening without proper access controls.
It might not necessarily be the developer who's at fault, my point is more that somebody in the chain knew the request was for S3 read access and the key was for FullAdminAccess.
At my job the alarm bells would be ringing and they would bring this up, but Infosys doesn't seem to have a culture that promotes that kind of security awareness.
I agree, but that tells me you are much more detail oriented than many developers I've worked with. Most devs at the 70-100k a year range are phoning it in 9-5 and only check exactly the boxes a PM and QA person make them check. I very rarely see developers who are in the standard deviation below median make that kind of check. These are good productive developers who get lots of tickets done.
In a world where all the problems are wrapped in containers and ever increasing bloat, it takes a lot of discipline to understand the stack, if that’s even the proper term anymore.
The Indian government has, over the years, awarded contracts worth billions of dollars to Infosys for projects like the Goods and Service Tax portal, Income Tax portal. In all these cases, the implementations are slow and super buggy. Deadlines to deliver are routinely missed. In an ideal world, these companies should not be allowed to exist.
Now, the interesting this is the recent complete refactor of the country’s income tax portal. It was messy, but I feel it was heavenly compared to the clusterF that was healthcare.gov. So what are your thoughts on this being a WITCH specific problem?
It was fixed because a govt minister threatened them with jail time in the end. That set them straight and they fixed the monstrosity to save their ass. Can't reveal more details.
That’s interesting to know, thanks lol. That’s the interesting part about corruption in india, in some screwed up way, it’s more “fair” than corruption in the US I think?
I notice the author is in London. For people in the US, my suggestion is to go instead to https://www.cisa.gov/report to report things like this. Especially when you're dealing with a company that doesn't have a clear vulnerability-disclosure process, or if you have _any_ concern about your own safety (physical, legal, etc.) when making a report.
Personally, I would have stopped right before "The Cleanup", and made a report.
Which option would I choose from https://www.cisa.gov/report? None of them stand out, so I would have chosen the last option, to send an email.
In 1999, I was an intern at a company in India. We wanted to put a machine in a datacenter, and the datacenter admin asked us to set the Administrator password to "password". Turns out that all the other companies that put their boxes in that datacenter did the same. Infosys was one of those companies.
Epic! Hope you don't mind my quote here. I enjoyed it :-)
"...I glanced over at the other boxes, and they all had stickers on them saying "Administrator/password"...The three of us from TSPL looked at each other, and our president told me to decide. I asked the datacenter guy why he needed that. He said that sometimes they need to shutdown the boxes so they can move them to a different power strip. I asked him if it would be sufficient to give him an account that only had local access and could only reboot the box. He thought about it for a bit and said yes... So I created a new account that required a physically attached keyboard for login, and all it had was the ability to reboot the box. Our app was set up to start up automatically on boot, so we weren't worried about someone having to start it. DC guy physically locked the box to a rack, showed us that he was keeping they key, and we headed back to the office...
...We now needed to test our setup, so we asked everyone in the office to let us use the internet connection. We tried accessing our app, and it worked!...
...Since I had Admin access to our box, I was also able to open the "Network Neighbourhood" of our box in the datacenter. On that network, I saw all the other hosts that were in the datacenter. They had names identifying them from India's largest IT companies. These were companies I'd initially though of interning at...I looked at our president and grinned, and he looked back and said, "Send me a safe summary report when you're done" and walked off to his office.
I double clicked on one of the other big boxes and was prompted for a username and password to connect to it...
In a world filled with more competence and less corruption, Infosys would have gone bankrupt 20 years ago. But here we are with Wipro, Infosys, TCS etc. all chugging along.
I don't believe software development is their bread and butter. Traditionally these companies are known as "systems integrators" which basically means they have armies of people who have RTFM for popular enterprise software products and will provide you with bodies who sit in your office and install or provide support for the product.
I think the coming recession will help that along ;) There's going to be less fraud and we'll find out who's been "swimming naked" when the tide goes out as Mr. Buffett likes to say. Like FTX..
Many years ago, I did some consulting work on a project that had been "delivered" by Infosys. It was, to put it lightly, a complete and utter mess in every way. Just from a security vulnerability standpoint, it had: SQL injection, plaintext passwords for user accounts, zero protection against URL manipulation, etc. And those are just the ones that come to mind immediately.
Lol, I love how he just opted to delete it. Great on ya for having some balls instead of walking on eggshells like most of these security back and forth dialogues.
Reading the recent posts about an Android bug and how difficult it was for the researcher to get them to fix and how he was reluctant to disclose or even threaten to disclose reminds me of a time gone past of… harder… type of hackers.
It’s like the completely backwards on the wrong foot.
In good old days you could do a lot without some massive Corp dragging you to court. Are you willing to risk years of self-funded courtroom process which may not turn out well for you, just to show how hard you are as a hacker? I don't think so. We know of cases where even reporting an issue caused lawyers to threaten you without any upside otherwise.
Britain's PM Sunak has Infosys connections via his wife and is also a CBDC proponent. If the dystopian future happens, we might look forward to security risks in addition to the privacy and state control risks.
Well, if they choose Infosys as a company to implement CBDC then at least we are safe for the next 20+ years, because there's no way they complete the project before that time :)
Governments already observe most every financial transaction, either because they run the money transmission system (ACH/FedWire/etc), or because countries with VAT send every invoice to the tax office. It's not clear how a CBDC is actually different from the current system, but insofar as it isn't different, it's not less private.
The GitHub user instead of reporting incident to their security team chose to take sneaky approach to remove the keys fearing the actions from company.
They will be fired and instead of retrospectively improving the security Infosys will ban all OSS contributions from their developers.
This kind of stories is one of the reason I visit Hacker News. Thank you!
It's funny and annoying to read every week or so about another epic fail of a multi-billion "multinational information technology company". Good luck with outsourcing your critical services and medical data to neurodivergents.
Yep, I guess you are right. But I think that some of those engineers come from Infosys-like companies, so they do exactly what they’ve been taught and what they are accustomed with. IDK, maybe it’s a cultural problem? Or is it a problem of following best security practices?
I understand that it’s useless to seek answers to such questions. Let’s leave it to philosophers. :)
Infosys is long known to be incompetent as an organization among people who have experienced their brand of greed and labor fraud; I highly recommend you avoid them as much as possible.
I've worked extensively with employees from these Indian labor companies throughout my career as a software-engineer. Many of my positions have implicitly been just to ensure that these developers can't do too much damage. For a long time I despaired about the poor quality of software these contractors would create, and wished there was some kind of justice for the companies that employ these bodyshops. Then I realised, this is how that justice happens. There's been a string of these high profile security incidents over the last few years caused by contractors from these labor companies.
One of my friend worked on WITCH’s treasury software used internally by government, from which millions of USD transactions happen on daily basis. Since remote access to server was not allowed, he used to use his personal USB drive to copy code from his personal laptop to the mainframe server on daily basis. With both being full of viruses.
I am so glad I did not pursue a job with them out of college, infosys that is. I told them I was no longer interested and they kept calling my parent's house and even tried recruiting my sister who was in HS.
How they go about stuff with that felt so weird, cause I would never get the same recruiter, makes sense they would do something like this.
They could have been doing this on behalf of a paying customer. They will sell anything with our without having any expertise. "Pay us, bring your data, and we will do the AI for you and help you".
I know a person that works as a "Cybersecurity consultant" at Infosys. Mind you, not in India. She got hired as an intern with no degree, after a few months of game testing experience and some Udemy courses. Then, got promoted after less than a year and her salary doubled. Good for her, but she openly admitted to me her best skill is Powerpoint presentations
If you've ever used a third-party outsourcing company like Infosys, Tata, Cognizant, or whoever you know you get literally nothing of value back for what you pay. Unfortunately, it's usually a cost-cutting movie that executives love and defers the pain to another day and creates another mess for someone else to deal with.
The breach notification to HHS typically comes from the covered entity. They often have the information on exactly what PHI was out there, how many individuals were impacted, and can provide the right info to HHS.
And with my experience in healthcare IT, I can say privacy and compliance officers take reports like this incredibly seriously. Those might not be the right people but getting an email to compliance folks inside the covered entity and saying “here’s a likely breach” will absolutely get the ball rolling.
PSA: In exposures like this, Contact the cloud provider too. They tend to have the right contacts for customers. And I'm guessing there are actions they can take as well.
The article mentions a takedown notice they received from GitHub instructing them on how to remove certain content from their repo. I'm guessing maybe this PR contains some of that content they were asked to remove, and there's a bug in GitHub when rendering a pull request page that references deleted content?
Every taxpayer in India is dealing with Infosys as of today. After years of delays and fucks up you still see texts like “xyzObj.ststus.label” right in the tax portal when you’ve logged in to check your tax return status.
Is it possible to do a full sweep across all tokens in all Python files (for instance) in Github and find such keys? Can you tell from the contents if it's a key or some such "important" string?
GitHub already offers this - they scan all the code that gets uploaded to look for keys. I think the issue here is that the code wasn't on public GitHub, but the artifacts were uploaded to PyPi
Yep, and if you don't look for them, you can be darn sure someone else is looking for them. I heard about an incident from a friend where a GitHub repo was created accidentally public (ran out of private repos and I guess the failure mode back in the day was just make it public) and that repo had developer level access keys in it. Some enterprising fellow was scanning public repos for this, grabbed the keys, opened thousands and thousands of the biggest GPU machines they could get on AWS and started mining bitcoins. They were nice enough not to delete production to make room for more bitcoin miners.
That's not nice, that's just smart. Delete production, and someone will notice right away. Leave production as it is, and they might not notice until the bill comes due.
The keys here were actually in the published package, not in GitHub, as it seems it was published by accident.
Here[1] are the prefixes used for all AWS IAM access keys. Here[2] is the API definition for an access key. If you're going to search all of PyPy for keys, here's some more keys you can look for: [3] [4]
Heh, your 3rd link must have OCR-ed a PDF or something because "(A3T[A-Z0-9]" is for sure wrong; I'm guessing they meant "ABIA" and then the 4th link must have copied from #3 (based on the commit date) because it makes the same mistake
Interesting that pull requests 1 and 2 on that repo cause a 500 error when I try to view their diffs. Is this due to action on github's part to try to suppress displaying of the keys?
Information security at Infosys is implemented through a combination of technological and organizational measures. The company has a dedicated information security team that works to identify and mitigate risks. Technologies used to protect data include encryption, firewalls, and intrusion detection systems. Organizational measures include employee training on security policies and procedures.
Complete failure by team to not see super user permission as risks
> intrusion detection…
Clearly the did not implement AWS CloudTrail threat detection otherwise when op accessed the account it should have raised alarms, so its just plain lie
> …training on security policy
So the GitHub user probably skipped those considering them boring. And instead of reporting their own failure chose sneaky way to make it go away hoping no one will notice
Can you please stop posting unsubstantive and flamebait comments and otherwise breaking the HN guidelines? You've been doing it repeatedly, unfortunately, and we have to ban such accounts.
If other people are wrong or you feel they are, you're of course you're welcome to post substantive comments explaining why. Other users in this thread have modeled that quite nicely: e.g. https://news.ycombinator.com/item?id=33635659. But snark, name-calling, and swipes are not ok—they're not what this site is for, and destroy what it is for.
Hilarious. Infosys is a known "mass recruiter" in indian colleges. WITCH (Wipro, Infosys, TCS, Cognizant, HCL) companies is where talent goes to die. No competent employee stays in those companies (from what I've witnessed). Wouldn't be surprised if this turns out to be just the tip of the iceberg, because putting people with 6-12 months of programming / computer "experience" (that they only signed up for because of the money) in charge of major production systems is a recipe for disaster.