I think the fact that software "engineering" (and, yes, those are meant to be scare quotes) doesn't have the same level of rigor of other engineering disciplines is what is truly at fault here.

Most other professional engineering disciplines have clear, codified rules of ethics. If the Chairman of a public company told a building engineer "Can you get rid of a few of these bolts on this beam here? I don't like the way they look." he would rightfully be told to get bent by literally nearly any structural engineer. And it's not because structural engineers have vastly greater morals or thicker backbones. It's because they know they could, and should, lose their engineering license (not to mention be personally liable) if something were to go awry due to a clear violation of engineering standards.

Kudos to you for sticking up for what was right, but it's still an overall process failure that this situation required you to have this backbone instead of being able to fall back on "What you're asking me to do is a clear violation of professional standards that could cause me to lose my license."

I took an Ethics for Engineers course as an elective in college (sometime during the stone age). Wrote a paper on Therac-25. Had I not, I may not even been cognizant of the risk involved.

A single week lecture on this topic could move the needle.

Or regulators like FDA could demand to see corporate training materials given to software "engineers" (I concur on the quotes) on how to promote product safety.

My school required a software-focused ethics course in order to graduate with a CS degree. It was great—one of the few courses that I lean on on a regular basis in my work.

Good on you for taking it as an elective, but it's weird to me that any degree in any topic can be accredited without having a mandatory ethics course, let alone a degree in a science/engineering field.

Would love to hear some specifics from the course that come up for you at work!

I'm glad Ethics for Engineers wasn't an elective but mandatory when I was in school.

Agreed and thanks! I actually didn't realize that tweet reply to your original reply was also from you - it's what got me thinking about clear engineering codes of conduct in the first place.

I sat through a very similar class once upon a time.

I think you're totally right. But I also think: If an alien visited our planet and saw the same decision being made, its reasonable that they may ask: Wait, why can't you just do this in software? Is your software really that bad?

Point being; the software industry is deeply dysfunctional; and the best people who aren't steeped in software every day can do to influence their decision making process is to trust and listen to the people who are. Software engineering is as much Engineering Software as it is Navigating The Absolute Dysfunction of our Software Landscape; actually now that I type that, I think its significantly more the latter.

I don't know if anyone is trying to fix this, or at least do better. I really hope so. I've been on major teams at three systemically critical internet companies and its not getting better; its getting worse. It scares the living daylight out of me that the biggest thing holding everything together is PagerDuty.

> Wait, why can't you just do this in software? Is your software really that bad?

Fly-by-wire in airplanes is pretty good, but it still has its problems, and a lot of verification work goes into it, probably more than embedded Windows.

In some jurisdictions, software engineer is a protected job title that requires professional licensing (ever wonder why Google only hires "developer" in their Waterloo office but "engineer" elsewhere?). IIRC the engineer that wrote the code for the Volkswagen scandal was jailed for failing his professional obligations.

For OP's case case, I'd assume/hope a professional engineer would need to sign off on this change or be able to tell the VP to f off.

Hardware came from oversees and there was no licensed PE on the US side. But you're right, there should have been.

This was before the device was submitted to the FDA for approval. Chance they would have rejected it. A fallback position for a developer not willing to Play It Hard would be to make sure the risk docs submitted to FDA document this failure path. There is a whole process you are supposed to go through to analyze risks / harms and whether your controls are adequate.

Back when I was studying Computer Science, the fact that Computer engineering is not engineering was a very hot topic. Engineering comes with certain guarantees. If a you don't put a load more than X on a bridge for the next X years. It will not break.

How can we offer that guarantee as a profession when hardware, drivers, OSes and even libraries that we use change and shift without our control and sometimes consent?

Software "engineering" will definitely need to be looked at differently from physical, but there are still things we can do.

Critical systems (or at least the critical subsystems) must not allow any unapproved/untested changes. Hence why it's better for safety controls to be hardware/firmware and not part of a general-purpose OS.

It's one reason why you see separate payment terminal hardware on self-checkout kiosks. The payment hardware is more tightly controlled whereas they can modify the kiosk much faster.

There are also RTOS (real time OS) that offer execution time guarantees. Used in aerospace.

The labor pool of software engineers is very diverse in terms of backbones, ethics and morals. It puts replaceable engineers in an awkward spot to object to unprofessional requests.

For an anecdotal example, I have been told that standing up for ethics in software engineering would have negative consequences on my performance (implied: bonus, promotions, career). I have left that company, and they probably found someone more agreeable to replace me.

I don't know how we can force ethics onto companies, especially large corporations who receive thousands of applicants into engineering roles each week. There do not seem to be good incentives for engineers to be overly concerned with the overall ethical impact of their work. And so there are many engineers who won't ready to replace those who would.

> The labor pool of software engineers is very diverse in terms of backbones, ethics and morals.

This is exactly my point. The same is true for other engineering disciplines, too, but by codifying their ethical responsibilities (and, in some cases, assigning liability to engineers who forego those responsibilities), other professional engineering organizations help to ensure a bare minimum for what is and isn't allowed in their profession.

I'll give another example that has much less dire consequences than the OP's. 90% of "scarcity marketing", e.g. "Act now! There are only 2 rooms left!" or even "8 other people are looking at this property!", is complete and total bullshit. I've even seen A/B tests where they developers were like "yeah, the data here is not real, we just want to see if it has an effect." Why is this even in the realm of acceptability? There is no gray area here - it's not just a "dark pattern". It is 100% outright lying. Yet I never heard someone stand up in those product reviews (myself included, so I'm no hero either) and say "How can we spend so much time on our 'company values' when this is obviously bullshit and slimy?"

I wish there were a "software engineering code of conduct" that said that outright lying to end users is verboten, and that software developers can be held personally liable if they are aware of the lie and still implement it.

As others have noted, professional licensure (which can be revoked) is used in other industries.

Not feasible to ask this for every developer, but for safety critical systems it should be mandatory.

What happens in other engineering disciplines is the government revokes your license if you make a decision that violates your discipline's standards. Companies can't generally just force their engineers to do irresponsible things because even if they fired you and hired someone new, that new person would be putting their own future employability on the line by conceding. Better to get fired from a bad firm than to have to find a new career.

Obviously this doesn't solve all the problems, but it works as well as any solution I can think of.

The caveat when it comes to software is that coming to a consensus of what the standard procedures and policies should be would be nearly impossible. If and when software joins the licensed engineering fields, a lot of people are going to be very upset at whatever the requirements end up being.

The caveat is very true, and I think that I would be unhappy if a licensing body told me I'm suddenly "not qualified" to do the work I've been doing well for many years. Besides, it wouldn't be just for me to bear licensure costs (whether direct or indirect by being out of work while I get licensed) to correct an ethical fault in companies.

Perhaps licensure could be an effective solution, but one that is not very empathetic to engineers. Maybe some kind of a government-owned ethics controller/body to handle unethical software would be more just for engineers. Although it could also be very inefficient.

The profession and the state have to choose between freedom and regulation - or some hybrid compromise. For computer types, the profession and the state have opted for freedom. I'd say that on balance it was the right decision. For other professions (doctors, layers, accountants) that balance has been achieved and codified over may generations.

For egregious ethical violations, the whistleblower act provides a remedy.

Both the ACM [0] and IEEE [1] have published Codes of Ethics, but I agree they have little teeth.

[0] https://www.acm.org/code-of-ethics [1] https://www.ieee.org/about/corporate/governance/p7-8.html

I am very suspicious of this post.

First, there's your story. To describe it as vague would be an understatement. What was the medical device? If the device does something important for the patient, wouldn't a big red button also be dangerous? You wouldn't want it turned off accidentally. Also the machine needs a button or switch to turn on, right? So why not have that component be the physical switch? And if the kill switch is on the touch screen, wouldn't that take up valuable pixels that could be used for other purposes? It would also make accidental triggering more common.

And you claim the button removal request was from the Chairman of the Board of Directors of a different company, who was a major shareholder? The board of the company you worked at, the C-level executives, and management all caved to him but you didn't, and you kept your job? That makes no sense. Someone with that much influence should have no problem getting an individual contributor let go.

Then there's you. Your Twitter account was created in September of 2017, but either you never tweeted until September of 2022 or your tweets auto-delete. Maybe you changed your username because neither archive.org nor archive.is have any history for your Twitter handle.

Your domain[1] was registered on September 22nd of this year, and it contains some general-sounding activism-du-jour. One might even call it "current thing". If anything it seems quite contradictory. Liberty does not march abroad... except when fighting a proxy war in Ukraine? OK, I guess.

I know my comment is controversial, but your post is exactly what I would write if my goal was to create perfect HN bait. It seems much more likely that someone lied on the internet than that things actually happened as described.

1. https://ldnma.com/

I don't believe I said the Chairman was from a different company. He was from the same company. He certainly could have had me fired, at the expense of destroying their schedule. As I stated at the outset, I left out details as to not identify the company (although there is enough here for a former colleague to put it together). Medical devices are a small world with often only 2 companies seriously competing per type.

Your due diligence data is largely correct and I complement your research. I repurposed this twitter account from one I created in 2017 and used as read-only in the intervening time. Also worth pointing out my HN account is quite new.

Wasn't great to cross the wires of my advocacy and technology, but I used the account I had. Didn't expect my twitter comment to be noticed.

I didn't promote my blog at all in this post, but since you mention it, if you read it more carefully I think you will find I take care to distinguish the Ukraine conflict from prior US interventions. Namely, one is a defense of a democracy which invited us whereas others (Iraq, Afghanistan) were not. US troops are not marching into Ukraine. It's even less direct support than the French provided the US revolutionary war.

I don't take issue with your questioning except - the presumption that someone is lying before they have even had a chance to respond to your information.

Edit: removed redundant sentence

Wow, the game is afoot.

But why?

I'm a deep state employee of an as yet unacknowledged three letter agency whose propaganda blog received no traffic from this page until grandparent posted the link. I am a master strategist, playing chess in dimensions string theorists only dream of.

"So I have just one wish for you—the good luck to be somewhere where you are free to maintain the kind of integrity I have described, and where you do not feel forced by a need to maintain your position in the organization, or financial support, or so on, to lose your integrity. May you have that freedom."

- Richard P. Feynman

I’ve found that more often than not the problem in these kind of situations is too many layers of communication between the boss making the decision and the people tasked with implementing it. Those in the middle aren’t in a position to say no because they’re not in the know about the details.

The solution is to talk to arrange to talk to the boss directly, even if you’re several levels away in the hierarchy. Sometimes it’s not worth it, but in a situation like this it clearly is. 9 times out of 10 a situation like this can be resolved with a sit down over a coffee.

Agreed, although I'm confident that everyone present in this particular situation would say that approach was asking to get fired. I was fortunate to be in a position where I didn't care.

But why did they want to remove The Big Red Button?

Sorry if that’s a dumb question.

Not at all a dumb question. But the answer will be.

The Big Boss Man hated the way it looked. He once spent 30 minutes telling a graphics designer what a worthless piece of sh* he was where everyone on the floor could hear. Corporate equivalent of a terrorist.

> There was a Big Red Button acting as a physical power cut-out.

> Request was to remove the physical button and rely only on a software (touch screen). Death was a remote harm, but moderate injury / user stuck in device / panic attack was plausible.

Good on you for not removing it.

I work sometimes in mineral processing (above ground | below ground) with conveyor belts, crushers, etc.

Big easy physical power cutouts are everywhere:

Conveyors have "rip cords" running alongside, pull them and you can stop a 30 tonne/hour conveyor maybe before someone drawn into the rollers is crushed to death or maimed.

Crushers | screens | most big equipment has single OFF knife switch with "gang plates" that take multiple padlocks - if you and two others work inside a crusher you each put a lock (three in total) through the gang plates and nobody gets to restart the machin until everybody is out and has removed their padlock.

Big physical switches save lives.

It’s not really important to the story but it sounds like someone with a lot of power and a knack for micromanaging got an idea into his head and wanted to see it through. The motivations can be many, including non-evil but fundamentally misguided ones, such as saving some $$ on the BOM or perhaps something as silly as making the UI more “uniform” with all touch screen interactions.

Unit cost probably.

That's a nice story... it's also possibly true.

But that's not how things work in most medium-large companies. I was asked to implement some telemetry in our client applications that I considered was going too far (clearly designed to get more information than was strictly required for feature function). I said no, manager said ok and assigned the task to John. John said sure boss and come end of year also got a promo. The telemetry is still there, now piped into a "personalization" feature; all above board (ToS-wise) of course.

Been in a similar situation where customer wanted me to implement supporting fishy, I asked my boss to confirm by email but he never did and told me to do it orally in vague term. I just didn't do it and told them to find someone else, they did find someone.

Always keep in mind that when shit hits the fan they throw your name very quickly and you are going to stand in court to defend yourself. That's what happens when you work on devices which can kill people.

I learnt that they cannot reproach you to stay ethical and they know it is very difficult to make someone do something which goes against his values, it is easier to find someone else.

Yea it's a red flag when someone only verbally responds. A countermeasure is to send a nicely worded email summarizing the discussion and inviting any corrections. But unless someone could be seriously injured, I'd just find another job. Not worth it.

Post title inspired by Doctor Who S09E12 "Hell Bent": https://youtu.be/p5eHJmQg-_M?t=44 A glorious tearjerker.

In the USA FDA would need to approve such a change for a device that could possibly kill the patient. And would be unlikely to say yes.

I once designed a hardware state machine out of TTL (no CPU, except a network interface that could only passively read and transmit the state) to ensure that a pressure vessel was safe (and this was ahead of the mechanical safety devices). Regularly someone would ask why we had this large device and didn't just replace it with a tiny MPU.

This occurred before the device was submitted for its first 510(k) approval.

Also, companies have some discretion in what changes they re-submit to the FDA and what the decide to just document internally. There are guidelines, like if you change the OS you better resubmit, but there is a wide swath of gray area. The FDA is not looking at every code/hardware change request. They audit your process.

Edit: typos

Yeah, I'm on the small molecule side but did a combo device years ago and was shocked at how slack the device rules were compared to the drug side.

Agreed. Class III hardware is well controlled (i.e., implants), but software (any class) and lower-class hardware is loco. My device was Class II.

This change would have certainly increased overall risk so a new 510(k) would have been required if this device was already commercialized.

And had your management not backed down, and tried to push this through as a Letter to File, you'd have been well-within your rights to blow the whistle to FDA. There's not a FDA employee alive that wouldn't go "hold on there chief" to a device change of this magnitude.

Damn. And I thought I was tough for holding the line against spam and popup ads at a 1-developer shop from 2001-2004. Thank you for your service.

You're still my hero

Similar situation (different magnitude, but still) that I saw on twitter the other day: https://threadreaderapp.com/thread/1589700721121058817.html

Engineers have some power to prevent harm. Use it when it matters.

In one case I came across a premature engineering sign-off to confirm production readiness could be averted simply by asking the manager (who himself had the required credentials) to sign off on the change himself.

Imagine a touch screen e-stop in a machine shop.

One of these days someone will try it. Spank them.

Software can be so buggy. It's a shame that the higher ups didn't recognize that.