Hacker News new | past | comments | ask | show | jobs | submit login

This is also an interesting contrast to the (I assume higher quality) kind of work being put out by their other engineering departments- how can they make other sound engineering decisions on the rest of the car and it's manufacture, but fail at the software?

Is it something specific to software vs. traditional engineering in Japanese culture, or is it something inherent to software in general? Given the parent's complaints about workplace culture in general, shouldn't the entire car be badly designed?




I worked in new auto development at Toyota (Japan) for four years in mid 2000s, right before this happened. The unintended acceleration problem has also perplexed me, because in my experience any safety issue was always given top priority. But I did experience limited familiarity with software/firmware. Usually that was looked at as part of the component, for example the engine control module in it's physical plastic housing. Remember the production auto will have Denso (or similar) delivering that ECU to the factory as an "assembly" and hundreds or thousands of these are assembled into the car and function together. At the end of the assembly line, various tests are run on the system and vehicle, but these appear to be of the "input/output" that someone else described. So why isn't software done better?

I'd say one problem is the testing process is not robust enough for software/firmware as it is for mechanicals. The tests revolve around driving the car, and variations on driving, and have evolved over years and decades. Heavy software control is newer, and in my experience was often not fully understood. Chief Engineers tend to come from mechanical. And there are always hundreds of problems to solve and limited time, and the black-box nature of firmware makes it hard to develop tests for.

A development review meeting tends include various mechanical departments, like Drivetrain or Engine or Chassis, many times they bring physical prototypes to the meeting and the problems and proposed solutions are comprehensible. The build testing procedures subject the car/parts to various mechanical tests, and long-lead tests are performed which stress parts till breaking. It's something you develop experience with an can get your head around.

Firmware bugs are not like that -- having just gone through this with my current consumer electronics startup I experienced the black box component first hand. I remember one software based problem we reviewed at Toyota, where something happened like: when the car has been sitting for >3 hours at <50F weather, and the ignition is started when the car is at a 15 degree angle (side of road) and driven on to a flat plane within 15 seconds, the "check engine" light turns on. Even though we had no test for that, we heard about it for customer complaints came through. But the fix is to have the supplier (in this case, but could be the Toyota engineer) fix that problem, with most of the discussion centered on timing and certainty of fix. I don't remember any discussions about global variables or lines of code.


my perspective from some friends that bought a Japanese software startup and moved over there to help run it a few years ago:

promotion is based on age and length of employment. employment is essentially for life. you can't really be fired and you if you quit you may never again be hired (and i can't imagine whistle blowing would bring you any better). seniority reins supreme and unquestioned. there is a huge focus on hard work and long hours but also on loyalty and face.

contrast this to where and how you're working and think about the differences it would make. my friends were constantly frustrated that, try as they may, they could not elicit feed/push-back from anyone that worked under them. part of this is them being stupid foreigners that can't "read the air" properly, but part of that is people being extremely hesitant to touch anything that could be considered a break in seniority or disloyal. and this was a start-up. these were the f'in cowboys in comparison.

from what i gather, this system worked out pretty good with the long, patient and deliberate development cycles in hardware. when a product cycle takes ten years it might not be that bad to have people running it 'cause they've been there 10-20 years. but it doesn't work very well in software.


Exactly why I'm commenting so much on this. I always tell my team "Pay attention to detail, quality, method. Be more like the Japanese", to now see that even the Japanese (and their quality poster-boy no less) have trouble applying their quality philosophy to software. Not sure where that leaves my advice.


It doesn't change your advice. Toyota has a well-deserved reputation for quality in a lot of areas: customer service (they've been replacing frames on trucks for a few years at no cost to the consumer), construction, reliability, maintainability. That they have flaws in one area just means that they need to apply their practices to that area too.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: