Hacker News new | past | comments | ask | show | jobs | submit login
Aviation Safety Whistleblower Report on 737 Max [pdf] (senate.gov)
235 points by sohkamyung on Dec 15, 2021 | hide | past | favorite | 124 comments



"Under the leadership of Chair Maria Cantwell..."

opensecrets.org reports that Cantwell's #4 lifetime donor is Boeing (Microsoft being #1 by a good margin). No conflict of interest here I suppose... Also, searching the report for 'director' is interesting as there's no mention of the role Boeing's corporate board played on exerting pressure to get the 737MAX to market in order to avoid Airbus getting a ton of orders, right? [1]

Another interesting issue is how the software testing took place at Boeing, another subject that seems to be lacking in the Senate report. According to reports testing was highly outsourced to low-paid temp workers [2].

> "Bloomberg's Peter Robison reported on June 28 that Boeing and its suppliers outsourced some of its 737 Max software development and testing to temporary workers. These temp workers, some of whom were recent college graduates, were employees or contract workers for Indian tech firms HCL Technologies and Cyient Ltd. Some of the testers and developers made as little as $9, the longtime engineers told Bloomberg. Former Boeing flight controls engineer Rick Ludtke said the move to outsource was centered on cost-cutting."

I suppose asking some of those $9 an hour employees to testify before Congress about their roles in testing would have been just too embarrassing to Boeing. Cantwell instead wants to push the blame onto the FAA for lack of oversight, thereby protecting Boeing's corporate board members from responsibility.

[1] https://www.nytimes.com/2019/03/23/business/boeing-737-max-c...

[2] https://www.businessinsider.com/boeing-outsourced-737-max-re...


> For instance, Mr. Ewbank cited gaps in the “Changed Product Rule” which allowed FAA to certify the 737 MAX according to dated airworthiness standards, like those for flight crew alerting systems.

So this is why there exist Boeing 737 Classic, Boeing 737 Next Generation, and Boeing 737 MAX. Avoid the need to redesign, in spite of changing regulation.

The Boeing 737 first launched in 1967. It is preposterous to use 1967 standards, after discovering through bloodshed more about what is dangerous and what is not.


Actually Boeing did design a newer twin-engine narrowbody plane - the 757, which launched in 1982. But (for reasons I'm not entirely sure of) it was never as successful as the 737. As Wikipedia says, already in the planning phase "the shorter 757-100 did not receive any orders and was dropped; 737s later fulfilled its envisioned role" - I guess the 757 was more expensive to operate, and airlines already operating the 737 didn't want a mixed fleet? So after the airline industry downturn caused by 9/11, Boeing decided to discontinue the 757 and go on with the 737. But I wouldn't put the blame only on Boeing, its customers had the choice and chose the cheaper 737...


I read that the low ground-clearance of the 737 was really popular since it allowed easier/manual luggage loading at smaller airports. I suspect their existing stock of spare parts would also favour a refresh of the existing model compared to a new one?


And training. Training is massively expensive. You gotta train pilots, crew, mechanics, baggage handlers, etc, etc.

The more situations in which you can say "the new one is just like the old one you know" the better.

Part of the reason pilots weren't initially properly trained on the MAX8 was because Boeing and the customers were winking and nodding at each other and trying to perpetuate the farce that the new plane was the same as the old plane and giving the pilots minimal training was part of that. If everybody knew how the new MCAS system worked there would likely have not been any crashes but that couldn't happen because it would have created a paper trail that indicated the MAX8 was pushing the limit of what was allowed under the old FAA cert.


It's not quite like that.

It's not that 1967 design is unsafe, nor that the MAX design is unsafe.

The problem is that 737 pilots were allowed to fly MAX max without recertification. A recert would cost a lot for the airlines, so Boeing pushed the idea that MAX is a drop-in replacement for the vanilla 737.

The whole idea was to make a more efficient version of the 737 with no substantial changes in flight characteristics. As it turns out, the change in behaviour is substantial.

The foul play is Boeing pushing the regulatory agency in the US around, and the agency succumbing to it.


I think it is reasonable, until shown otherwise with a new type certification, to presume the MAX design of larger fans pushed forward is unsafe. Flight characteristcs are different, though currently covered up by software.


This company priorized cost over safety. That's who they are and finding shortcuts around new regulations is exactly what I'd expect from them.

I will never let myself nor anyone I care about fly on a 737 Max.

Even if I could be confident in the plane, I cannot forgive nor forget all the people who needlessly died.

I've boycotted fast food chains for far less transgressions.


Boeing killed my friend, Anne-K. Feigl, on that Ethiopian flight


Hey I'm very sorry to hear that. Sad way to go, I hope you find peace with it


  > I will never let myself nor anyone I care about fly on a 737 Max.
I feel the exact same way. Note that they are no longer marketing it as a "Max". You've got to look at the winglets and the engines to be sure.

  > Even if I could be confident in the plane, I cannot forgive nor forget all the people who needlessly died.
Actually, there are arguments for and against being able to be confident in the hardware itself. The engines are in the wrong place, simply. They engineered computer-aided pilot assist systems to compensate for that. I've heard pilots and engineers weigh in on both sides of the debate. But everyone agrees that a plane like that would never be designed from the ground up. It is literally a stack of patches.


737 Apache


There are ~600 delivered MAX aircraft, and ~4700 still on order (including all the cancelled orders).

I don't know where you live and how often you fly and for what purpose, but, chances are, a lot of people be somewhat forced to fly on a 737 Max eventually.


Why would anyone be scared to fly on the most scrutinized plane in history? I certainly don't care. All of the pilots flying the MAX would now receive even more in-depth training on the quirks of the new systems.

Chances of critical incident are practically zero. 180 countries have re-certified the plane and design changes were made as well.

If you don't trust the FAA or the 179 other authorities, how do you step on ANY plane?

Additionally, you don't trust the MAX yet you trust Boeing (you didn't mention their other planes)? If Boeing is negligigent, it's unlikely it's limited to a specific product. How can you then trust maintenance programs and spare parts provided by Boeing for the Dreamliner for example?

If Boeing is incompetent, trusting the Dreamliner but not the MAX is irrational.


> All of the pilots flying the MAX would now receive even more in-depth training on the quirks of the new systems.

One of the main issues was that, apart from an hour on an iPad pilots didn't receive training at all.

"even more training" is disengeneous and makes you sound like a shill. I'm sorry to say.

E: spello


Pilots are trained in dealing with stabilizer trim runaway. The first crew to encounter the MCAS problem followed it (turned off the stab trim system), and landed safely. The second crew, on the same airplane, did not and crashed. (They never turned off the stab trim system.)

http://knkt.dephub.go.id/knkt/ntsc_aviation/baru/2018%20-%20...

Boeing followed this up with an Emergency Airworthiness Directive, which was distributed to all MAX crews. It included a 2-step process for recovery.

The EA crew did not follow that procedure:

"Initially, higher control forces may be needed to overcome any stabilizer nose down trim already applied. Electric stabilizer trim can be used to neutralize control column pitch forces before moving the STAB TRIM CUTOUT switches to CUTOUT. Manual stabilizer trim can be used before and after the STAB TRIM CUTOUT switches are moved to CUTOUT."

https://theaircurrent.com/wp-content/uploads/2018/11/B737-MA...

This does not absolve Boeing of the design flaw in MCAS where it only read data from one sensor, had too much authority, and would repeatedly engage after the pilot countermanded it.


You obviously know way more than I do about this, so maybe I completely misunderstood but what I saw on one of those YouTube videos by a 737 pilot was something like that before MCAS when you got a runaway trim situation and followed the procedure and got trim back to normal and had the trim system turned off, most of the time you could turn the trim system back on and the problem would not come back. Runaway trim was usually due to some transient problem that would be gone after the system was turned off and on.

In those cases where the problem does come back it is not any worse so you can just try again. You might go a whole flight doing this if you are getting enough time between the problem recurring for restarting the trim stab system to be less annoying than not having it.

With MCAS, if I understood correctly, it would add a bias to the trim that was not reset when you restarted the trim system. Each cycle of turning it off and back on that was harmless pre-MCAS would increase that bias until you reached a point where the trim down bias was more than you could counter.

This wouldn't matter for the flights that crashed, because they didn't get to the turn it off stage, but it seems likely it would have eventually happened to some crew that did follow the procedure including the emergency directive.

The impression I got from this, and from watching many episodes of "Air Emergency", is that there is the official way to operate a plane as documented by the manufacturer, and then the unofficial way that pilots actually use that comes from combining the official way with the pilot's mental models of how the systems work to cover what can actually be done. So if the official way says to turn something off, but does not say "and then keep it off until a mechanic checks it out", and the pilot's mental model says that it is safe to turn back on, then they might turn it back on.

MCAS invalidated the mental model pilots had for the stab trim system, but the MAX-specific training they got and the emergency directive after the first crash did not do anything to tell pilots that their model was invalid.

When documenting a system and training people to use it, you really need to take into account the mental model they will have of the system. It is their mental model that people actually use to guide their interaction with the system.


The second flight totally did get to the turn-it-off stage. They turned it off multiple times. Unfortunately, during that process, they neglected to manage their throttle, leaving it at near full takeoff thrust the whole time, and reached a very high speed. Thus, while they could turn off MACS, they were unable to manually restore trim with MACS off due to the strong physical forces on the stabilizer. There was probably a recovery path for them where they lowered speed, then corrected the trim manually, but they didn't see it.

Some day, there will be an AdmiralCloudberg write up on it, but he hasn't done it yet: https://www.reddit.com/r/AdmiralCloudberg/


In the Internet era, I can directly access reports from leading experts. Why would I be interested in a reddit user?


Because he does a pretty good job of producing nice summaries. Why not both?


I have no idea if this person knows what they are talking about, either in total or for any particular bit of information. Also, why create a filter between me and the original report - a great benefit of the Internet is disintermediation.

Personally, I find original reports from experts to be much more clear than the attempts of others to digest them for me. To write clearly, you need to know what you are talking about. The better you know, the more clearly you write.


I mean, that's all well and good, but assuming that your pilots will correctly diagnose and correct a failure of a new system that may or may not have similar symptoms to existing emergencies is still a really poor practice. If MCAS had been designed to command a continuous forward pitch moment until the AOA excursion had resolved, it would have very accurately resembled a pitch trim runaway following an AOA probe failure. As it was, it clearly didn't have a strong enough resemblance.

Safely operating a poorly designed aircraft can be done, but it starts with explaining the deficiency in exhaustive detail in a bold typeface in a prominent place in the operating manual, with clear warnings to avoid certain flight regions, and a well-documented emergency procedure. For example:

MCAS FAILURE: If you experience repeated momentary uncommanded nose-down pitch excursions. 1. Conduct RUNAWAY PITCH TRIM procedure. 2. Do not reset pitch trim circuit .

Unfortunately that would have required new training.


Runaway stab trim is very easy to diagnose. The airplane pitches, the two big wheels on either side of the console start spinning, and there's a loud clacking sound.

The MCAS failure exhibited as runaway trim.

> it clearly didn't have a strong enough resemblance.

I disagree. The trim randomly and repeatedly coming on and driving the pitch down is runaway trim.

> a well-documented emergency procedure

Like this one distributed to ALL 737MAX crews:

Boeing Emergency Airworthiness Directive

"Initially, higher control forces may be needed to overcome any stabilizer nose down trim already applied. Electric stabilizer trim can be used to neutralize control column pitch forces before moving the STAB TRIM CUTOUT switches to CUTOUT. Manual stabilizer trim can be used before and after the STAB TRIM CUTOUT switches are moved to CUTOUT."

https://theaircurrent.com/wp-content/uploads/2018/11/B737-MA...


>Like this one distributed to ALL 737MAX crews:

...after their poorly designed system caused a plane crash.

If you have a 737 type rating and can speak to type-specific training standards I would love to be educated, but the defining characteristic of runaway trim failures that I've experienced is that the trim keeps going in one continuous motion until it can't go any further or you manually shut off the system. A momentary, uncommanded attitude change would initially make me want to troubleshoot the autopilot, rather than the trim system. This is exactly why you have to describe new systems and their failure modes in detail, even if they have elements in common with and use the same emergency procedures as existing failures.


I do not have a 737 type rating. But I did work on the 757 stabilizer trim system and gearbox design for 3 years. I know what runaway stabilizer trim is, and have been through the failure mode analysis on the 757 trim system.

> the defining characteristic of runaway trim failures that I've experienced is that the trim keeps going in one continuous motion until it can't go any further or you manually shut off the system

Runaway trim is the trim coming on when it isn't supposed to. It could be continuous, it could go it fits and starts, it could come on randomly. The corrective action is the same - turn it off.

This is why the trim cutoff switch is prominently there on the console within easy reach.

Waiting until it can't go further, i.e. it runs into the stops, is just not a good idea as by then the airplane may be in an extreme pitch position which may not be recoverable.

> ...after their poorly designed system caused a plane crash

The LA crew never turned off the trim system, despite restoring normal trim with the electric thumb switches 25 times.

The previous LA flight experienced the same MCAS malfunction, and after restoring trim a couple times, turned off the stab trim system. They then proceeded to their destination and landed normally. They did not know about MCAS, but they did know that runaway trim is dealt with by turning it off, which is a memory item.

The MCAS system was poorly designed. But a contributing factor to the crash was the pilots not following proper procedure in response to runaway stab trim.

I am not a pilot, so take the following as you will:

1. if I suspected an autopilot malfunction, I would turn it off and fly manually and let the maintenance people figure it out.

2. if I experienced runaway trim, I would turn off the trim and fly without it as much as possible, again letting the maintenance people debug it.

In general, I am not going to debug a flight critical system at 30,000 feet that is malfunctioning if I can fly safely without it.

I agree that Boeing made a big mistake in not disclosing the existence of MCAS and how it operated.


In all seriousness, I would enjoy talking about commercial aircraft trim system failure modes over a beer sometime.

For what it's worth, while I agree with your technical definition of a trim runaway, every time I've seen it in the sim or real life it's been a single, continuous event moving from steady-state flight trim to an extreme. I'd be willing to bet a few beers that this is what most pilots are trained to expect from a trim runaway, and what B737 crew see in the sim while getting type rated. I'm not disagreeing that if the LA crew diagnosed it as a trim failure and performed the EP correctly they would likely still be alive, and I'm also not arguing that they were an exceptionally good or even average crew.

I'm arguing that you can't really fault a below-average-but-still-acceptably-competent crew for not diagnosing the failure of a system they couldn't have reasonably been aware of as a trim problem on an otherwise perfectly functioning aircraft. There are plenty of atypical emergencies that require the crew to "do some pilot shit" to get the plane back safely on deck, but an easily foreseeable single-sensor failure shouldn't be one.

We'll probably just have to agree to disagree about how likely an average crew would be to treat this as a trim failure, but I like to think we can still agree that the likelihood was unacceptably low for commercial aviation safety standards.


I don't mind at all having a friendly disagreement. No problem!

I can't really imagine erratic operation not considered as a failure. After all, if you're coming in to land you wouldn't want the stab trim coming on uncommanded even for a second. As far as the 757 Flight Controls group was concerned, an intermittent failure in the trim system was unquestioned cause for immediately disabling it.

Two independent computers controlled the automatic stab trim. They were custom computers, designed by two groups that weren't allowed to talk to each other. They used different CPUs, different algorithms, and different programming languages. The computed commands were run through a comparator. If they differed, both computers were instantly electrically isolated from the trim system.

How Boeing evolved from that ethos to relying on a single sensor, I cannot understand.

BTW, these ideas have trickled into my approach to writing software, often engendering spirited debate with me against the world :-)

I claim credit for the term "defensive programming". It was the title of a talk I gave long ago. I'd never seen the term applied to programming before, and have seen it often sense. Unfortunately, I have since lost the contents of my talk. I don't even remember which conference it was at, there have been so many.


I work on the avionics side of the industry and really enjoy when I run into your posts in discussions. You explain things to people not in the industry much more eloquently than I could.

While I'm not in my company's fly-by-wire group currently, I have been in the past.

> Two independent computers controlled the automatic stab trim. They were custom computers, designed by two groups that weren't allowed to talk to each other. They used different CPUs, different algorithms, and different programming languages. The computed commands were run through a comparator. If they differed, both computers were instantly electrically isolated from the trim system.

Current thinking in fly-by-wire software is a little different. There have been studies performed that showed nearly all software issues at this level are due to a misinterpretation of requirements. These misinterpretations were shared between the different software teams, leading to the two different units outputting identical (though incorrect) commands which would pass through the comparitors. So in essence you're doubling your development cost for no actual safety benefit. I can see if I can dig up those studies if you'd like. It will take a while, though, since almost everyone at my company is already on vacation for the year.

I'm simplifying what follows a little as I'm not sure how in depth I can get on our hardware design. What we do now is essentially run the same fly-by-wire software over multiple computers. These computers must have a mix of CPUs, including having differing endianness. If a single computer miscompares the comparitor turns that computer off. If more end up failing, the system falls back to a much simpler failsafe mode without a CPU in the loop where the flight controls in the cockpit are interpretated directly by the electronics that drive the actuators.


Thanks for the information. I can understand misinterpreting the requirements - after all, the requirements themselves are a form of programming, and getting the requirements clear and bug free is a major endeavor.


Heh, I think I did a poor job of expressing myself. You're absolutely right that an intermittent/erratic trim operation is a failure, and potentially a serious one. I'm thankful that you, as a flight control engineer, are as concerned about it as you are (for hopefully obvious reasons). Out of curiosity, how probabilistic is your failure mode analysis? I'm wondering what the relative likelihood of an intermittent uncommanded actuation compared to the neutral-to-extreme runaway failure that most pilots expect. I've never been in a sim where the operator console had trim failure options other than "stuck at current position" or "runaway to extreme limits," but I wouldn't be surprised if they should've included other failure modes.

Yeah, that seems like an eminently reasonable way to design a trim system. I don't think the MCAS concept is fundamentally unsound, but it blows my mind that they didn't design it with that kind of mindset.

Oh man, I wish more software was built to the standards of aerospace control system best practices...


> probabilistic

I don't remember the numbers, but one had to show that the likelihood of failure was less than 1 flight in XXXX where XXXX was a very large number.

> I wish more software was built to the standards of aerospace control system best practices...

I am much more ambitious. The Deepwater Horizon disaster, the Fukushima nuke plant failure, the Toyota acceleration failure, etc., are all designed with an utter disregard for lessons that aviation learned long ago. To wit:

1. a failure of one subsystem shall not propagate to another (zipper effect)

2. a failure of one subsystem shall not compromise the whole

3. once any such failure is detected, that subsystem shall be assumed to be compromised by demons, and must not be allowed to continue

4. you cannot educate people into not making mistakes

5. if you punish people who make mistakes, they will conceal mistakes instead of being forthcoming about fixing them

Those principles have strongly influenced all my engineering work since, and I've tried to propagate them among my colleagues (with mixed results).


For example, the good old `assert` in software. Asserts are to detect impossible states, and hence if an assert trips, the software is in an unknown state, and what it will do next is unpredictable. Therefore, an assert must go directly to jail, not pass Go, and not collect $200.

I get pushback all the time on this,

1. people use assert()'s to validate input

2. people insist that their program can continue operating after an assert trips

3. people insist that their program cannot be allowed to fail, and that they are capable of predicting what their program will do next


And again, victim blaming instead looking at the real perpetrator that is Boeing


It is the pilots' responsibility to READ, UNDERSTAND, REMEMBER and FOLLOW any Emergency Airworthiness Directives sent to them. Flying an airplane is not a joke. Pilots who are unwilling to do this should turn in their pilots' wings.

Crashes are usually due to a combination of factors. Boeing certainly shares considerable blame for the poorly designed MCAS system.

More issues:

1. how did a defective AOA sensor get past checks and be installed?

2. how did a defective AOA sensor pass inspection tests on the airplane?

3. why was the LA flight allowed to fly, when the previous flight experienced stab trim runaway?

4. why was the LA flight crew not informed of the anomalous behavior on the prior flight?

Air travel is made safe by addressing all factors that led to an accident.


The second design flaw is manufacturing stretched airplanes in the first place, which are hacks in general. Even as an amateur one can see that the proportions are off without understanding anything else. They look ugly and fly ugly.

The stretched cargo version of the MD-11 had many issues and accidents, particularly during landing.


> an amateur

While there is something to the notion that if it looks like it will fly it will fly, remember that the general configuration of modern airplane design took many, many years to settle down on. Even the Wrights got it wrong (putting the stabilizer in the front made it unstable).

The notion of swept wings, that seems so natural today, was a very long time in coming, for another example.

For a third, the aerospace people keep finding ways to make the wings more efficient. Notice the winglets on the wingtips? Those are fairly new.


I don't work for nor hold stock in any airline or aviation manufacturer, I can just adequately judge risks.

You are the one who thinks 180 separate, independent regulators colluded to protect a company at the risk of the people they are supposed to protect. Any argument against the MAX has to be against the regulators as well as Boeing itself.

Why is the Dreamliner acceptable and the MAX is not? Same company manufactured and maintains them, after all.


> Why is the Dreamliner acceptable and the MAX is not?

I remember [1?] hearing that Emirates threatened refusing 777s & 787s from one of Boeing's plants due to repeated quality control issues. Have not tracked the issue.

> 180 separate, independent regulators colluded to protect

Not necessarily. But if 180 _independent_ bodies were effectively independent, one of them would probably have found issues with the max. But they are not operating effectively independent.

[1] https://www.reuters.com/world/middle-east/emirates-doubts-fi...


The plant with the quality control issues is the non-unionized one.


Different airframes. Different issues here.


I think he means more training now.

Not really disingenuous at all.


What’s remarkable is that MCAS functioned so well that not a single pilot had noticed it prior to Lion Air.


Why would anyone be scared to fly on the most scrutinized plane in history?

Finding a bug is not the same as fixing a bug, and finding lots of bugs doesn't mean you've found them all.


So I imagine your position is that the software industry should swear off products altogether when they have a serious bug, right?

Did you make a conscious effort to rid yourself of OpenSSL after Heartbleed? After Heartbleed, OpenSSL received mountains of extra funding as well as unprecedented scrutiny.

Guess what happened after the MAX disasters? Extra funding and unprecedented scrutiny. In fact, no plane has been scrutinized more.


"Most scrutinized" means that you might have a fair chance at finding a higher proportion of them, though.


That's not what was being talked about. Sure, it's the safest aircraft out there, but that's like saying "the safest car". No-one is going to go and seek out the safest car, maybe stay away from unsafe ones, but as long as the general level of safety is high, people will fly on anything.

The problem here is just simply ethics. Boeing did a bad thing, therefore I will refuse to use this plane. If enough people think similarly, Boeing will be punished harshly by its clients, as airlines see that the plane is not profitable due to its public image.


The problem is not about if the MAX is the safest airplane now. If Boeing can get away with it like that then it lowers the general airplane safety in my opinion. Now that Boeing has designed an airplane by cutting corners to safe money (even though it did cost them a lot, the government saved them), then this means that other airplane manufacturers must also cut corners to stay competitive.


You seem to be labouring under the illusion that airliner manufacturing is a competetive industry. But i'm not aware that Boeing has any real competitior other then airbus. The phrase "to big too fail" looms very large and is very real here. So no, this won't have an adverse affect outside of Boeing. The real problem is the culture that allows such a disaster and then, worst of all, tries a cover up. With the full aid of a government regulator (for heavens sake!). To the entire world. Apparently not afraid that being caught out would have worse consequences. A massive public breach of trust and hints of corruption. The way the FAA was the last country in the world to ground the MAX, it makes the US look like some 3rd world dictatorship.


> If Boeing is incompetent, trusting the Dreamliner but not the MAX is irrational.

The 787 has its share of problems, but at least it's newly designed and has to conform to the latest safety regulations, not an over 50 year old design like the 737 MAX. If it wasn't such an old design, the odd placement of the engines which couldn't fit under the wings and therefore the "workaround" for the issues caused by this odd placement (MCAS) would not have been necessary...


Boeing has never made an airliner that was fully dependent on computer assistance. They have no heritage in this domain and apparently don't care enough to pay for capable testing. It will always be risky to fly these no matter how much scrutiny is applied.


Indeed, my wife already has had to. Apparently it is uniquely uncomfortable to boot, even for a short (5'2") and thin (110lb) woman. I'm 6' tall and dreading the day I have to fly on this.


Depends on how they configure the cabin, but it's been the same as any other 737 whenever I've flown on it (5'10", 180lb)


>Whistleblowers like Mr. Kucera provided information indicating systemic problems continue to exist, including understaffed FAA offices charged with certification oversight responsibility for manufacturers and the continued risk of undue pressure under the FAA’s system of delegated authority.

>According to whistleblowers, GE Aviation’s GE9X engine program suffered from undue pressure on production staff acting on behalf of the FAA. For example, Mr. Kucera described being placed in an untenable position where he was responsible for conducting engine conformity tests on behalf of FAA, while also being charged with preparing GE engines to pass these same tests.

Wow.


Once upon a time, being an FAA engineer was a prestigious career move. As government work has become more and more stigmatized, so too has the brain drain from the public to private sectors been more and more pronounced.

I used to work in airworthiness and I met some really outstanding people working for the federal government, but they were dwarfed in number by the teeming masses of crayon eaters. Sadly, it's quite hard to get fired from the public sector, for a litany of reasons beyond the scope of this bitter internet comment.

I never worked for Boeing or GE but I'm not at all surprised to read those allegations. The ODA program was designed for the same type of world as FTP. To wit: the world was a different, less adversarial place back then.


I couldn't help but laugh out loud at the crayon eater comment. Hilarious and I'm sure in many cases it's not inaccurate.

That said, I personally question whether all gigs have to be prestigious and cutting edge and super interesting. I think there should be plenty of room in government and elsewhere for qualified people who do do a good job, but might not be the 20 something work 80 hour per week and rise fast type.

I wish more organizations openly admitted to and positioned themselves that way - like, hey, are you good at your job but don't really want to work super super hard all the time and rise to the top of your organization, or feel like you always have to work on the most prestigious, most cutting edge projects? Then we're the place for you!


I’ve been in a similar boat to the grandparent. The issue with the government (and more broadly government workers) isn’t that the work isn’t “exciting” or some such.

It’s a cultural and institutional resistance to change or improvement and the rewarding of toadyism and punishment of innovation.

I firmly believe the government would be “better” in some ways if it was forced into transparency by managers being allowed to fire for abusive reasons, rather than the psychological torture they regularly inflict on high-quality employees that don’t tow the line. Instead you either leave of your own volition or you are cowed into silence, becoming complicit in a general miasma of less-than mediocrity.

Brain drain is a natural conclusion, but only helps people who are both smart and motivated. Folks who are motivated and not smart get crushed and folks who are smart and not motivated get pushed intentionally into irrelevance.

The federal government is the most toxic employer there is, and they gaslight you about their toxicity by being so willing to assign you to BFE and let you languish rather than firing you, as if firing is the worst thing that can happen. Amazon is a well known toxic environment for tech workers and is a massively better work environment for an engineer than the federal government.


How much aeronautics/airworthiness/safety R/D is actually conducted by the FAA? My understanding is that they used to do a lot more cutting edge research, the kind that would attract engineering talent.

Looking at the 2022 FAA budget[1], they asked for $159M for "Engineering, Development, Test and Evaluation" out of a total budget ask of $18B+. That's less than 1% of the total budget.

1. https://www.transportation.gov/sites/dot.gov/files/2021-05/F...


> How much aeronautics/airworthiness/safety R/D is actually conducted by the FAA?

Virtually none. It's widely recognized that airine mfg. technology is a rapidly moving target, and the FAA is not in the airline mfg. business.

The FAA is in the business of collecting and reviewing certification data, which is why the fake MCAS paperwork is a really big deal.

NASA operates the X-planes R/D program, not the FAA.

https://www.nasa.gov/centers/armstrong/images/X-Planes/index...


It's not about prestige as much as budgets and "public/private collaboration", ie prior to Reagan and the "small government movements" most regulatory bodies had well funded labs and engineering teams capable and authorized to do independent research in order to validate claims made by industry.

post Reagan when the government budgets were cut and organisation asked to streamline/"rationalize" outsource and cut down on fluff the first thing cut were those labs as the new way of thinking was that they should let the regulated do their own safety research and that the government should merely check that the resulting documentation was conforming to formatting standards rather then validating the results.

This causes situations where large companies basically gets to be both accused and judge in cases where the safety of their products are in question, as they are the only one who can realistically produce and present studies to the regulators on the safety of their products. And it's not limited to the Boeing, nor to government as the same ideas of "rationalization" is prevalent in the management textbooks most enterprise managers are trained from.


Part of the problem with government work prestige is that for every FAA engineer, for every NTSB investigator, you get three or four - for the lack of a better word - political commissars these days, whose job is to blindly carry whatever the higher-ups deem "important" at the time down the ranks. Something about rotten apples spoiling the bunch.


> As government work has become more and more stigmatized ...

> Sadly, it's quite hard to get fired from the public sector ...

Aren't you stigmatizing it?

IME, it's quite hard to get fired from large corporations or any large institution, except for management adjusting 'human resources' counts in order to cut costs, or if you do something politically 'wrong'. One person I know left a Fortune 100 engineering job - a manufacturer you've certainly have heard of - in frustration over that issue. They simply did not show up for their job and it took ~3 months to lose it, IIRC.

The civil service is designed to our government from political maniuplation and firing. IME, the standards of professionalism are very high, higher than businesses.


Stigmatized? I assumed people just preferred getting paid more and possible working at interesting companies.


Here's the thing, the FAA is no longer a new prestigious institution that is growing. All the ingredients for smart and talented people to succeed and rise to the top. The FAA was founded in 1958. It's now an ossified institution with middle managers, bean counters, process and a litany of other bullshit that talented people shouldn't tolerate and that mediocre people can hide behind.

Process is all about risk mitigation. You add process to mitigate the damage that can be done by a single individual. Sadly though when you can't have process that only neuters the liabilities. Processes also neuter your assets.

The only attractive thing about government jobs these days is the guaranteed pension. The guaranteed pension is also the exact kind of thing that causes many people to be overly zealous about not rocking the boat. It becomes all about staying in your job at all costs until you've put in your time and can retire worry free.


>I assumed people just preferred getting paid more and possible working at interesting companies.

There's that too, but at least in my circles there's very much the attitude of "oh you're in the public sector because you couldn't hack in in the private sector where you have to perform".

Whether that's a merited attitude to hold is a separate conversation, but the sentiment definitely exists.


Surely that attitude just flows from the salaries being so low. If the pay was good people would understand why you were in the public sector.


It's self-reinforcing in both directions, I'm sure.


This sounds like a normal test engineering position if the operation of the test and preparation of test article has been delegated to that team. Engineers should be empowered to work honestly and tell their company if a test has been failed.


Now throw sales teams, middle management, executive pressure and competition in this equation. These things push engineers to cut corners and find workarounds. This is true for all software engineering it’s just that sadly in case of airplanes it costs lives.


Michael David Crawford wrote about airplane software and computers and safety but was arrested by the police because he wrote it at Grove Beach McDonalds and was charged with being a nuisance. Soon after these plans like Flight 370 started to happen. He was the first one to discover the bugs in Boeing's software and never got credit for it, just a year in jail with time served.


It doesn't really seem like his year in jail was related to any bugs in Boeing's software that he discovered [0]. Do you actually have anything that suggests that?

His site was archived. For example this [1], but I can't find anything about what you've suggested.

[0] https://news.ycombinator.com/item?id=19488164

[1] https://web.archive.org/web/20190326051854/http://www.warpli...


He was writing the article at McDonald's and the police arrested him for saying "If I am not permitted to save this document, an airplane might fall out of the sky." when they wanted to arrest him for being homeless in the McDonalds. It was all on Kuro5hin and K5 users sent him money and items and books in jail.

https://news.ycombinator.com/item?id=19489570

He published a lot of stories and articles at K5.


> He'd headed south from the Portland area, wrecked his car in SLO, and was arrested for being a nuisance at McDonald's -- probably for helping himself to free ketchup.

> They dropped the charges and released him the day after my jail visit. He'd been locked up for several months, also spending time at the Atascadero mental hospital

I don't see anything about that, at that link - it provides a very different explanation for the arrest.

Nor do I see anything similar to your suggestion at any of the linked backups of K5 posts inside that link.

So again... Is there anything to confirm this claim, at all?


No nothing left to confirm this claim. Anything that does has been removed from the Internet.


There are (incomplete) K5 backups, with posts like this one[0], which happened around the time he stopped posting. Do you recall any details at all that would point anyone towards finding what you've suggested?

[0] https://atdt.freeshell.org/k5/story_2012_2_24_05715_2076.htm...


The backups are incomplete on Kuro5hin and several of his websites. Anything that might prove what I wrote was not backed up. He was also several nicks like Mr Markup and Repeatable Hairstyle. Because Rusty would ban him and remove his diaries because Rusty got C&D letters from lawyers about Crawford's posts. Then Crawford just came back as another nick.


This [0] is the incident report from the McDonald's. And this [1] is the referenced website post by Crawford. This [2] is the still intact place where he wrote under the pseudonym of "Jonathan Swift" (among others).

The McDonald's incident report clearly points out he believed an officer had a personal problem with him, and points to where he did a write up on it, which would not happen if it was just a cover up. It makes no mention of programming, let alone anything else. Nor do any of his writings after his release talk about planes.

Whilst his other writings are somewhat hard to follow at times, and there are places where he makes claims such Al Qaeda not existing as an entity, there is nothing about planes anywhere.

In point of fact, the only place at all, in any of the archives, or the circle of people who frequently wrote around him, the only place I can find mention of planes is this HN thread, that you have written.

I believe at this point I have been beyond generous at researching, but the simple fact is it was never related to Boeing's software.

[0] https://www.scribd.com/doc/81649240/Crawford

[1] https://web.archive.org/web/20120531183316/http://www.softwa...

[2] https://groups.google.com/g/softwareproblem


[0] is not from McDonalds but in Portland when he lived with his mother. McDonalds was in Grover Beach California. [1] Solving the Software problem was a human issue not a machine issue. Bugs in software are created by human beings and management tries to cover them up. [2]Trolls took over that group and began posting sci fi porn on it.

You are mistaking but one incident in which he got arrested in Portland, OR with the Grover Beach, CA McDonalds incident. Using it as a strawman.

Mike was harassed by cops because he was mentally ill. There are several incidents of that. The McDonalds report was buried I wonder why?

So who are you, a Beoing Schill?


Crawford would write the truth about megacorporations like Drobo and Rusty Foster would get C&D emails to remove those diaries or posts. One of his nicks was Zombie Jesus, add that one to the list.


Not all comments and stories were scraped: https://atdt.freeshell.org/comments/2012/2/24/05715/2076?pid...

Does not work, it was a comment by Mumble. Proving that the archive is incomplete.


a) Yes, incomplete means incomplete. I said that.

b) The links are broken because they take a different URL form in that particular backup. All the links are broken.


> It was all on Kuro5hin and K5 users sent him money and items and books in jail.

I don't see that as evidence of truth, whether Kuro5hin or any other internet forum. Think of the things people believe and do on the Internet.


The News Media didn't take him seriously because he was mentally ill. So he had to post on Kuro5hin and other places on the Internet.


What does that tell us about the truth of what they are saying? How do we even know that they contacted media organizations and what the response was?

> News Media

There's no such thing. Look at the incredible, diverse array of news publications. Compare it to operating systems or social networks, for example.


Boeing is exactly the type of company that would engage their army of lawyers and relations in the Justice Department to put a "nuisance" behind bars.

Someone who endangers the holy profit for something measly as flight safety is a prime target for companies like Boeing, Nestlé and all the other fundamentally evil corporations out there. The pretty detailed Wirecard documentaries and books out there are a useful perspective into the world of these corrupt people.


I'm not denying that Boeing is a company that behaves badly, at all. The senate report into the 747-Max shows that they actively manipulated testing conditions, and pressured engineers, so that they could get certification when they knew the plane could be unsafe. Which is unforgivable, and should constitute a whole heap of crimes.

However, that a large aeronautics company would hound a known schizophrenic when it's much cheaper to dismiss them as nuts (and extremely easy to paint them as such), is another thing entirely. And when you're acting out because your symptoms are flaring, you are very likely to become a public nuisance.

Without a single scrap of evidence for one, and some for the other, the statistical likelihood lies in one direction, and not the other.


If he had an episode in McDonalds it is against the ADA to arrest him. He was painted off as a lunatic to discredit him. It is like arresting a diabetic for insulin shock and calling them drunk.


>Michael David Crawford wrote about airplane software and computers and safety but was arrested by the police because he wrote it at Grove Beach McDonalds and was charged with being a nuisance.

This reads like he was arrested for writing about airplane software, when it appears he was arrested for having a schizophrenic episode in a McDonald's and was likely asked to leave and refused.

>Soon after these plans like Flight 370 started to happen. He was the first one to discover the bugs in Boeing's software and never got credit for it, just a year in jail with time served.

How exactly did he get the software in the first place?


Is there a link to that story?

Isn’t that a 1A violation?


It was on his website because he self published, he killed himself a few years ago and his website is down forever. https://twitter.com/soggymike?lang=en

Because he had schizoaffective disorder nobody took him seriously. He has a degree in physics and tons of experience in programming and software engineering he worked on Mac System 7.5.X as a contractor for Apple. His name is in some Easter egg on some version of it.

It is a 1A violation, but nobody cared about it at the time and he only posted to Kuro5hin which is down forever as well now.

It worked they shut him up and hid the evidence.

Here is an interview with him on CNN to prove he exists and CNN sees him as an expert on software engineering: https://youtu.be/pv9_M_a0uIg It is on YouTube because CNN banned it.


> CNN sees him as an expert on software engineering

CNN has zero expertise in engineering.


Don't let the paranoia overwhelm you. A lot of your conclusions regarding this man are bordering on delusional. You might be catching what he had.


You cannot catch schizoaffective disorder it is rare and affects 1% of the population.


You don't need to be a schizoaffective to develop paranoid delusions about being persecuted by hidden forces that don't exist.


I am having trouble finding anything about this; Google results bring up nothing. Can you provide a source?


All sources are downed websites that have been taken off Google. It has been covered up. The main space for it was Kuro5hin which was taken down a few years ago.


[flagged]


what ?


Edit: is there a place that you can read a tldr, and what this actually "means" ?

Anyone here an expert? How safe are these planes and their derivatives if I fly frequently? IIRC, they made some updates to the new planes that uses the same engines. Also more training for the new models? Would those be enough?

Also how do you prevent using specific airplanes as a passenger. Some locations, airlines, and their nearby cities i think only uses these planes and their derivatives right? So how do you avoid that? Seems like driving for multiple days is a terrible idea. Would you prefer using multiple nonprimary airports and small engine planes?


The only update that would be dependable is adding a third AOA indicator to implement voting logic. Boeing is too cheap to do that and the FAA is too gutless to force them.


> 97 pages, geez, is there a place that you can read a tldr?

Like most such reports, it starts with a section called "Executive Summary". Maybe start there...


Thanks! will definitely keep that in mind next time i see a white paper


We in the academia invented it. We call it an Abstract and we put it at the top.


Abstract and executive summary are not the same thing. https://www.umass.edu/buscomm/absumm.html


I as a frontend web dev. just awed in how do you even begin to program an airplane


You don't program "an airplane". Each box has its own spec to which it must conform, so for example the little controller that regulates fuel flow into the combustion chamber is first modeled as a system in e.g. MATLAB. For throttle position X, at altitude Y, with air temperature Z and Mach number W, the fuel flow must be ABC pounds per hour so set the servo valve position to DEF.

So on and so forth (it gets a lot more complicated than that but you get the idea).

Then, you ask the mechanical guys what the operating environment will be (vibration, temperature, pH, etc.) and you source a chip from an approved manufacturer that can handle your system dynamics and the environment. Then you hit the autocode button in MATLAB, flash the generated code to the chip, and run the programmed chip through bench tests.


> For throttle position X, at altitude Y, with air temperature Z and Mach number W, the fuel flow must be ABC pounds per hour so set the servo valve position to DEF.

This is fascinating. So underneath it it's a big decision tree with a few defaults thrown in. Is that an accurate description?


Not sure about aviation computers, but for car engines it’s typically a set of tables with something like air pressure on the y axis and engine rpm on the x axis where each cell has a fuel value. Then you have other tables for other variables such as spark ignition advance, or anything else that can be changed electronically on the specific engine. Tables are typically used because they make the software quite simple/fast. And the values in the table can be tweaked (either from simulation or by experiments on the real engine) that separates the configuration from the code.


  > for car engines it’s typically a set of tables with something like air pressure
  > on the y axis and engine rpm on the x axis where each cell has a fuel value
You can see those values being edited here while tuning a Viper:

https://www.youtube.com/watch?v=93cOSZkKAvo


>So underneath it it's a big decision tree with a few defaults thrown in.

Not in the sense that it's yes/no decisions all the way down. Most of the inputs are continuous functions, and your output will be a function that combines the inputs in some particular way specific to the application.

You might do something like:

   if (mach > 0.5 && mach <= 0.85) {
      ...
   }
And so on and so forth


  > if (mach > 0.5 && mach <= 0.85)
But surely there's some else clause at the end with a default value, no? Or at the minimum log.error("No matching flight conditions configured", pressureAir, pressureFuel, airspeedMach, engineRPM) or such?


Yes of course there is fault handling! That's one of the hard things about safety critical sw is that you actually have to deal with everything.. Lettings things "fail" or be "undefined" isn't really an option.

E.g. Mach is derived from multiple sensors so you have a first layer of redundancy there, then if all fails (and you can detect it ofc..) you'd probably start some kind of dead-reckoning and issue WARNINGS accordingly to the pilot and/or connected systems that change its behavior to a more conservative one.

There is almost always some kind of "safe state" that wouldn't depend on the input signals at all. It would obviously negatively affect the performance of the aircraft and most likely reduce its operational envelope.

Pilot action would be required, i.e. reduce altitude, reduce AoA, reduce air speed etc. LAND ASAP etc. :)


Terrific, thank you!


I don't know how it's done for planes, but I've heard that many automotive ECUs are programmed graphically, as a block diagram. The engineers basically pretend it's an insanely complicated analog system with amplifiers, filters, switches, and what have you. Then the whole thing gets compiled into a real time binary that simulates the system one timestep at a time.


That is correct for a lot of aircraft systems as well, especially in the design/requirements phase the actual implementation might vary of course. I've personally used Simulink a lot for control systems.

I.e. hydraulics, fuel transfers, ECS etc.


Yeah, when trying to diagnose issue with my car I found ECU documentation that contained suspiciously complicated block diagrams (with all parameters labeled with abbreviated German), which I'm not sure if they are just documentation or program source itself: https://i.imgur.com/BcW1ho6.png


Think of it more like a piecewise continuous model with a decision tree that chooses which model to use.


Much like you program any other computer. At my shop we use C, C++ and Ada. Some problems suit themselves well to modeling, so something like Simulink can be used (which translates to C++). What throws people off is usually the fact that these are hard real-time systems (requires a different mindset) and that some functions are distributed across multiple computers.


Hey there I teach people to code some of them I teach web.

I think the easiest way to go about this, is to learn how to fly a plane. Then once you've learnt that you can kinda learn it piece by piece. IE: it's easier to teach someone how to drive a car first before teaching them how to mod it and write a self-driving program for it.

But, then again, some mechanical engineers learn how to build planes, before knowing how to fly a plane.

It really helps if you can reach out to someone in that industry and they can show you the ropes


My first day on the job at Boeing, in the 757 flight controls group, as a newly minted mechanical engineer, my lead engineer told me to "size the jackscrew". The jackscew is what drives the leading edge of the stabilizer up and down. If it fails, the airplane becomes uncontrollable.

I went back to my desk, stewed for a while, and panicked.

So I go back to Erwin, shamefaced, and said I had no idea how to size the jackscrew. He chuckled, and said "did they teach you column buckling in school?" I said sure. He said it's a column buckling problem. You know how to do it.

And sure enough, I did.


Nothing to be ashamed of there. You hadn't had enough domain specific exposure to cut through the jargon from the sounds of it. First exercise I do with my hires/juniors is work to connect what we're doing with something they already know.

That's communication 101.

201 is divesting yourself of the assumption you always know exactly what everyone else is talking about, and remembering that it never hurts to play the fool and learn something, as in the process of being taught, sometimes new knowledge will become apparent.

People who master both pieces tend to be the most amazingly competent people I've ever had the privilege to work with.


OOh, ooh, ooh, me too!

A couple months into my first job, the chief engineer, who was an ME in an EE company, told me to figure out how a certain filter worked. I sat down at my desk with the diagram for hours struggling with it. Then I got up to go to the bathroom or something and came back in through a different door and saw the schematic upside down and that from that perspective it looked totally different. It was just a 2-port network; I know all about 2-port networks!

A few minutes later I was done :-)


10 PRINT "CLEARED FOR TAKEOFF, RUNWAY 21L";

20 GOSUB TAKEOFF;


Compartmentalization, is how.


Check out https://www.amazon.com/gp/product/B08P98854S/ref=ppx_yo_dt_b... for a great history of all this.


What we didn't hear about is how Boeing is going to change its processes to avoid repeating this nightmare scenario. Because, sure, they fixed the 737Max AFTER hundreds of people died and every airline was forced to ground their airplanes. Who can trust them when they come up with another plane? I, for one, don't.




Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: