Can’t believe Tesla would ship something with anything resembling a default password. At first glance, I assumed this would be a clear violation of the requirements of CA SB-327 (goes into effect Jan 1).
Reread the bill, and it actually says: “The preprogrammed password is unique to each device manufactured.” If the default is based on the serial number, I guess it’s “unique” under the letter, but certainly not the spirit.
This is amazing. It is all completely plausible, and clearly includes only the most easily explained horrors, with anything that would require explanation just omitted.
It is worse than I would have been able to invent.
IMHO, these stories aren’t unexpected from a tech company that grows 80% year over year, developing and expanding as fast as technology and market forces allow. Yes, there are mistakes and unfortunate incidents in the organization of people and tech priorities here, but this is inevitable in an engineering org that moves this fast. You really need to measure this up against Tesla’s achievements:
Growing high double digits each year for more than a decade in a brutally competitve space, developed 4 hugely successful vehicles, first US auto company to succeed (knock on wood) or grow to significant size (>700,000 cars sold) since Ford, production constrained continuously, did this with almost zero paid marketing, built global networks of dealers, service and chargers, improved the state of battery tech by a multiple and doubled global li-ion battery pack output, built 3 car factories, made electric cars economically viable, incidentally seriously wounded the car dealership model, succeeded with qualitatively different software approach with OTA updates and tight software integration, designed cars with hitherto unparalleled performance in their class, and others that I couldn’t come up with on the spot.
This is only controversial because this is safety-critical tech and it’s uncomfortable to see the development pains of consumer grade software affect it. There’s no evidence that the approach have led to more injuries or deaths than cars do in general, quite the opposite. Tesla cars are safer than competition, the argument can be made that fast deployment is a net win regarding safety. Even in the absence of any other things they have achieved.
I believe there’s also an element of envy, or misunderstood beliefs that the best tech approach is taking things slowly. Yes, in isolation. But then market forces will crush you because someone else was faster, or because you missed the window of opportunity for making things so dramatically better that you manage to unbalance the local maximum of the status quo.
[Edit: To everyone who downvotes this: For the sake of enlightening discussion, could you please express your disagreement in words rather than the downvote button? Note that I'm talking about GP's linked commentary on Tesla's engineering mishaps in general, not this specific vulnerability].
>For the sake of enlightening discussion, could you please express your disagreement in words rather than the downvote button?
I'll give it a try: maybe it's because someone made an off-hand comment about Tesla shipping a default password, and you chimed in with an apologetic post that reads a lot like astro-turfing?
>did this with almost zero paid marketing
Traditional marketing. Tesla spends millions on marketing annually, it's right there in their financials. Just recently a pile of pro-Tesla Twitter bot accounts were banned. Did you know that Tesla is paying people to post messages on social media?
I am not affiliated with Tesla or paid by them in any way, if you're implying that. Was also not making apologies for shipping devices with guessable passwords, that's obviously a serious screwup that needs to be fixed - ideally at a level of changing the engineering culture if that's what it takes. But the rest of this thread treats that subject in depth, and I wouldn't contribute anything material to that discussion beyond piling on, which is a bad use of time and brain power. Thought I made it quite clear in my comment that I was specifically adressing the discussion of sub-par software engineering practices, and how this is a very common consequence of extreme growth.
I know Tesla has a marketing budget and marketing people, that's obvious enough, so traditional marketing was indeed what I meant. Maybe I could have made that more clear.
Wasn't aware that they're paying people to astroturf though; first I hear of this. Do you have proof of it?
> Traditional marketing. Tesla spends millions on marketing annually, it's right there in their financials. Just recently a pile of pro-Tesla Twitter bot accounts were banned. Did you know that Tesla is paying people to post messages on social media?
Umm what? I can assure you that most pro-Tesla Twitter accounts are happy owners. I know cause I am one. The amount of FUD on Twitter directed against Tesla is insane!
I created my Twitter account in February 2008 and it has pretty much stayed dormant for the most part. Until I bought a Model S and discovered the TSLAQ trolls and often challenge their B.S. claims with actual sources. Case in point: https://twitter.com/teslahistorian
> Just recently a pile of pro-Tesla Twitter bot accounts were banned.
> Did you know that Tesla is paying people to post messages on social media?
Yes, while reading this initially I, too had the same reaction: Typical tech startup bootstrapping seat-of-the-pants stuff. But upon smelling the coffee, and remembering a few conversations I have had with ostensibly reasonable people, this sounds totally batshit.
>Growing high double digits each year for more than a decade in a brutally competitve space, developed 4 hugely successful vehicles, first US auto company to [...] grow to significant size (>700,000 cars sold) since Ford
People get in this product and they don't see the startup culture making wins and beating the odds to deliver a Ludicrous! Tesla Model S with heated leather seats. They see a finished product who's origin, to them, may as well have been a Star Trek replicator. To this point, I have encountered people who argued strongly that Teslas are "Self-Driving Cars" already. They really give stuff the benefit of the doubt sometimes.
So many issues attested to in that engineer's report need to have been already handled. It is amazing that they have created this manufacturing line out of nothing, I mean they build these things in tents, really bucking the odds here, but it causes me much pause. I would have hoped Elon would have worked out bootstrapping this manufacturing process, I think he discounted what it takes to pull off consistent manufacturing quality on many levels. This was Toyota's innovation a lifetime ago, shame that Tesla didn't seem to hire up any of that greying industry knowledge.
I now feel a deeper understanding of Tesla's icing-out of the repair/aftermarket. Seems obvious in retrospect.
>I believe there’s also an element of envy, or misunderstood beliefs that the best tech approach is taking things slowly. Yes, in isolation. But then market forces will crush you
>There’s no evidence that the approach have led to more injuries or deaths than cars do in general, quite the opposite. Tesla cars are safer than competition, the argument can be made that fast deployment is a net win regarding safety.
Get with the program. This is not an electric toothbrush with a subscription model, people are going to die. When product safety and manufacturing quality take a back seat to business whims, you are going to get trouble. Sporting this Boeing-esque reasoning is going to get people killed.
I think the problem is that the software model is being used in other industries. Not just in production (e.g. Tesla "move fast and break things" on things that are life/death, or merely very expensive) but also in valuation: Tesla, WeWork, and a ton of others are/were valued as technology companies even if they are clearly not.
When this happen you start getting comically bad results (see: all the summon feature failures on Twitter) and catastrophic valuation corrections (see: almost all the recent "technology" IPOs).
The whole VC market seems to be built on this built-in assumption that everything will be able to scale and have the margin of software companies, even when it is obvious that it is impossible. I am not sure why there is such a gap between reality and VC/investors in general, but it's probably going to seem obvious in retrospect.
Fanboism that kills is generally frowned upon, by all those who work in majured safety relevant industries.
We do hop all the safety hurdles every day, for very good reasons- and to call these proofen right processes slow by someone who lives only because they exist- it has a certain sting to it.
Speedy development is all fun and games, until your luck runs out and you toast a whole highway filled with familys in one afternoon.
After that, nobody is going to hold cheery fanbois liable who enabled this culture. No, it will be engineers, who where forced to make fast decisions. Remember Toyota-Bar - why dont you justify your senseless optimism, in the face of physical danger and accidents, to the victims of sloppy development?
Why is the burden of proof not on you? After all that blood at least should be on your hands.
Reminds me of a time when an ISP would provide their customers routers where their default Wifi passwords could be derived from their SSIDs. A free app would allow you to connect instantly to practically any Wifi network in the city.
An ISP around here still ships consumer routers with passwords consisting of a random 8 digit hex number, i.e. that could be cracked in a day on a GTX 970m (tested hash rate against my parent's wifi).
They also ship with default SSIDs, numbered 100 to 999, so given 900 GPU days of precomputation you could create rainbow tables that allow for cracking every default password/ssid pair.
I can tell you from the wifi passwords I have been given by friends that many people are not changing them.
At least you need to capture a handshake to use your rainbow table...
I disagree. The spirit of the law is to ensure that logins cannot be automated. Unless the serial number can be read over the internet without authentication, using it is completely within the spirit of the law.
>Unless the serial number can be read over the internet without authentication
As I said in my post below, I just checked mine and the full serial number is included in the device hostname. Since tons of regular PW installs are going to average joe residential sites which will heavily be using crappy outdated ISP provided routers with default passwords and entirely flat unmonitored LANs, possibly with infected machines to boot, it's fair to say that yes in fact the serial number will be able to be read over the internet without any authentication. Anyone who can see hostnames of devices on the LAN can get the whole serial.
Although I also doubt any such network will have any real rate limiting or notice any hammering either, and the serials look utterly trivial to brute force. So I'm not sure the fact that they're all broadcast for everyone even ultimately makes much difference.
I had a very similar thought regarding gaining API keys for Tesla vehicles after realizing I can get generate API key knowing only my Tesla Account user name and password only.
Honeypot free WiFi at a Tesla Super Charger with a legit looking login page: “Free WiFi for Tesla Customers, Login to your Tesla account to access.”
API End points include vehicle unlocking, speed limit settings, etc. Some are not available while the vehicle is motion, so at least there’s that.
Yes, but the only time that this would be an issue is if someone, somehow decides to install it themselves or the Tesla technicians installing it forget to change the password which is very unlikely considering it's part of the standard process that you have to sign for upon install. You have to choose your own password either way or accept that you didn't.
So you can change the password to the management portal and prevent access to the API / admin panel which provides access to all of the mentioned settings?
Exactly. If the SSID was “Password=BSSID MAC” and the password _was_ the MAC address of the BSSID device, I suppose it would technically be unique. This is barely more secure than that, IMO.
"The PW can store up to 13.5kW of electric power..."
No, you store energy, not electric power, and energy is Wh, that's why you have your phone battery with 5V and 8000 mAh as units and not 8A.
"How does the grid feel about me oscilating this and who will die first, the PW or the sub-station?"
Definitely the PW. Sub-stations are made to withstand thousand of households switching all at once their light-bulbs at evening and during storms nature electric discharge (ie, lightning strikes). Your measly PW is fully ignored.
"Do not try any of these Grid Codes (really, please do not):
AU ASS4777.2
DE VDE4105
UK GA59 21A, GA83 16A
IT CEI-021
NZ NZS47772
US: IEEE1547 Split Phase 240V 60Hz
"
I'm a bad boy, I tried them. Nothing happened.
If you were to try this and somehow cause damage to the sub station, would you be held liable?
Actually, this brings an interesting question with IoT - if devices damage or otherwise exploit your utility company’s equipment, is it their fault of yours? Presumably doing anything bad is against your utility’s ToS, but if you didn’t do it knowingly...
Those are commands for your PW, not for sub-station. Like I said, I tried them, and nothing happened. The author is an alarmist and the only valid point he had in the entire article was default password instead of assigning unique one, which is easily to re-configure. Also the last one I suspect might damage your PW ugly enough if you're in US, but I am in EU where I already have 240V, so ¯\_(ツ)_/¯
After authentication, the fact you can make it charge or dump power into the grid is by design. If I wanted the grid to suffer, I can do this by plugging in and unplugging a multi-kilowatt heater every few milliseconds too. Residential properties have a fuse (usually 60-100Amps), and anything you can do without blowing that fuse won't damage the grid.
> and anything you can do without blowing that fuse won't damage the grid
That's what you'd think. But I have actually done damage to the grid and had that fuse still in one piece. I accidentally connected the -HV line of a neon light installation to ground. After I could see again I realized the power had gone out. I called my boss (landline still worked) who lived more than a kilometer away from the workshop and there too the power had gone out. It turned out my little accident had put a good chunk of the eastern part of Amsterdam out of commission and the fuse was just fine.
Fuses react to current, not to power and when you dump a high enough voltage back into the grid your fuse might not blow before you cause something further upstream to trigger to protect the rest of what's connected.
This is exactly what RCDs are for - with a huge earth leak like that (and therefore L/N imbalance) it would have tripped within milliseconds, before cooking the grid.
I am surprised by how infrequently they seem to be used outside of the U.K. - I’ve just rewired my house in Portugal as it was completely unearthed, and just had fuses, no RCDs.
I'm not sure how common these are in an industrial setting nowadays, but back then all big electrical tooling would leak at least a little bit and if you hooked them up through ground fault interrupters those would trip all the time.
A typical lathe, mill, plasmacutter or welder (which that particular workshop was full of) would be in the 10KW range with peaks far in excess of that during start-up.
I don't remember if we had the office area (where I was working) on protected circuitry or not, that would have been an interesting bit of data. The building still exists, I'm almost tempted to go and have a look.
That HV supply was a pretty beefy one by the way, the crocodile clamp that made 'first contact' was gone entirely.
It's required on all new construction in Norway for a decade or so as well. Very common for renovations as well.
Norway also just switched to smart meters, which can detect ground problems as well, and supposedly a lot of issues has already been prevented by these being automatically reported. I suppose this is something which is more likely to spread to other countries, as it's not just a safety thing, but also an upgrade that makes your life more convenient, and saves the utilities money in the long term.
RCDs are common in Australia. In Western Australia, whenever a house is sold it needs to have an electrical inspection which includes checking if there are enough functional RCDs.
Not sure if that's a national requirement or not, but houses I've lived in in Victoria also had RCDs.
I'm sorry, but I don't believe one bit of your story.
Ohm's law dictates that it's be impossible for your HV generator (neon transformer) to outpout a HV voltage when connected to the grid (you'd need thousands of kWs).
Feel free to contact Riny Assink who was my boss / co-founder at the time for corroboration, I'm pretty sure he will remember because (1) it was a pretty memorable occasion in its own right, (2) the neon installation was intended to be used on a trade show the next day and my action meant we did not get to use one of our nicer eye catchers which really pissed him off, (3) he built the stuff that I blew up in the weeks leading up to the event, which pissed him off even more. I'm sure a bit of googling will turn up contact info for him.
As for Ohm's law, when dealing with large transformers made for 50Hz dumping the full charge of a (pretty beefy) HV power supply into the low end of a circuit looks roughly like a lightning strike would, only with a much lower amount of Joules behind it. So it's not so much Ohm's law but Maxwell. Chances are some protection circuitry said 'better safe than sorry' and disconnected.
FWIW when you receive a bit of data that contradicts your knowledge and assumptions in general the more interesting bits of knowledge are found by asking how things could have happened rather than to disbelieve a priori the facts as presented by eye witnesses. I have fuck all to gain from relating this story, and would happily stake my reputation on it being 100% the truth without any embellishment whatsoever. The whole reason I told it is because the person I replied to made roughly the same claim that you did.
I agree that it should not be possible, and yet it happened.
Charging or dumping a single powerwall will not damage the grid. However, if someone causes many of them to do so in sync, the current grid control systems are definitely not going to cope well with that sort of behavior.
To do that the attacker would need to connect to hundreds (thousands?) of Powerwalls at the same time and figure out the password for each of them. Which means being close enough to each of them to access the Wifi and also figuring out their serial number somehow. How would they do that?
Not feasible for a single idiot to accomplish, but you're forgetting we live in an age of State-Level Actors who distribute very sophisticated malware to attack enemy nation's infrastructure by borking SCADA controllers in e.g. uranium enrichment plants or petrochemical refineries.
One may hypothesize about the feasibility of a router-targeting worm (similar to the Moon router worm, targeting Linksys kit) that has a payload that searches for nearby PowerWalls and uses its owners's resources to play guess the password. Getting in wouldn't be fast, but over a period of months such a worm could well gain control over thousands to tens of thousands of powerwalls. At which point, a DDoS attack on national-level grid infrastructure is within reach.
It would be more feasible to just hack industrial controls or the power company rather than create some worm for a specific router and hope that you can find the 50k powerwalls.
> figure out the password for each of them. Which means being close enough to each of them to access the Wifi and also figuring out their serial number somehow. How would they do that?
Per TFA, the serial number is ST<AA><B>0001<CCC>.
AA is the year, for which there are currently 5 values (15 to 19).
B is the revision, for which there are currently 6 values (D to I).
CCC is provided as part of the broadcasted SSID.
So they would "do that" by enumerating and trying out the 30 possible serial numbers for each SSID.
Without the default password, an attacker would need to actually own all those powerwalls to pull that one off... At that kind of money, there's better ways to be evil.
The fast switching ability isn't a bug by itself, but it makes the insecure authentication much worse. Shitty IOT auth does not mix with power electronics.
Whether Tesla is worse than other providers in this space is another question, and may well have a depressing answer.
The fast switching is still a failure of defence in depth.
It could be internally rate limited to some value (which I don't know), and also with some random backoff value so Powerwalls could not be synchronised.
Tesla has full control over every PowerWall, which means a rogue employee at Tesla would be able control all of them simultaenously. Given what we know about security at Tesla, I have little faith that their internal systems are advanced enough to prevent this.
You don’t see any difference between an individual switching a multi-kilowatt heater every few milliseconds vs a malicious actor automating this attack on 100,000 properties over an entire state?
But if only the true owner of each powerwall had the ability to control it (when the default password issue is fixed), an attacker can't get correlated control like that.
Rate of Change of Frequency protection (ROCOF) in embedded storage and generation systems is what will kill the grid in a massive cascading failure.
ROCOF works well to keep things safe when only a small percentage of the grid demand is met by residential solar/powerwalls.
As soon as any significant proportion is residential solar (and thats already the case in some countries at some times of day) it acts as a cascading failure mechanism. As soon as any failure occurs, embedded generation sees a rapidly decreasing frequency, and rather that increase supply as traditional generators would be instructed to do to stabilise the grid, ROCOF protection requires they cease supply, making the issue far worse.
Within a fraction of a second, all embedded generation will disconnect, likely causing a near total blackout nationwide. Since system frequency that ROCOF measures is nationwide, failures won't be local to one geographic area.
I suspect these rules were made when people thought "consumers feeding energy back into the grid will never be more than 0.1% of the total - we'll always have enough spinning reserve to make up for that". Thats no longer the case, and unless the ROCOF limits are changed, and the majority of home solar/wind/powerwalls get a firmware update, expect a few very large blackouts.
I believe the solution to this problem is to ban ROCOF protection, and the related phase shift protection, and instead instruct a few big energy producers to transmit a gold code on top of the 50 Hz AC, bandlimited to 48-52Hz and power limited to 0.01% of the system power. Transmitting that code would be easy (cheap) for anyone who does DC/AC conversion with solid state electronics, so that's normally solar, wind farms, and long distance undersea transmission lines.
That gold code could be received and decoded anywhere on the network. If power islanding occurs, embedded generation will detect the loss of the gold code (since they are no longer connected to the generator injecting the code), and cease supply.
The only disadvantage is it introduces a security vulnerability by design: Anyone could transmit the gold code from their house, effectively disabling islanding protection in their neighbourhood. If power islanding were to occur, and if there was sufficient embedded generation to keep a stable power island, grid hardware could be destroyed through overvoltage, overheating, and circuits closing without frequency synchronisation. I think it's a worthwhile tradeoff though - damage will be localized and minimal, and a very unusual set of circumstances have to happen outside the attackers control for the attack to do damage.
A DC grid is likely better nowadays, but we're stuck with the historic AC grid. Most of the problems DC faced back in the war of currents have been solved by solid state technology.
When home renewable reaches a level of penetration where simultaneous loss would be in excess of spinning reserve any further embedded generation will be regulated in such a way as to not contribute to the problem. For instance by prohibiting exporting energy back to the system, and separating your house from the system if the frequency is falling quickly and your embedded generation is going to trip,in order to avoid the simultaneous load step due to the loss of all the embedded generation they could just disconnect it all. And in that case the embedded generation can continue to feed your house if it can operate an isolated network.
Isn't that the problem, that every home device will disconnect at roughly the same time? Leaving the atrophied generators to take up far too much slack.
Embedded generation will firstly displace local load before exporting any power back to the system. If the generation disconnects all you are left with is the load. If the frequency is falling and the ROCOF protection is going to cause the generation to trip, better to trip the load too otherwise the system just gets an increase in load.
> This is the mythical power-grid attack that people have been talking about since the concept of cyber-warfare was first dreamt up.
> It’s lucky we caught this now, before there are enough PowerWalls to seriously destabilise the grid if this attack were to occur.
I could be misunderstanding you, but do you seriously think that there are not more destabilizing attacks already available? From my reading the US power grid is already extremely vulnerable to attack.
Any power grid is very vulnerable to attack. Anyone who can cause a sudden surge in demand can take a power grid down.
If you can make power usage unexpectedly go up by more than ~10% within a minute, most power grids will fail.
I'm struggling to think of any companies who could do that though... Someone with malicious access to teslas servers couldn't even do that... For example, instruct all plugged in tesla cars to start charging all at once. Assume 400k tesla cars are plugged in overnight, with an average 10kw charge ability. That means tesla could add 4 gigawatts to the grid demand instantly, which is only 0.4% of the USA's ~1TW generation capacity.
I'd assume that the power grid is much more vulnerable than that at least some of the time in some regions. What if due to natural load variations the network already is close to the capacity reserve (which could be measured by tracking AC phase) when somebody mounts an overload attack?
What happens if the attacker generated a (e.g. periodic) pattern of load changes that excites control mechanisms at their resonant frequency?
Maybe the book "Blackout" [1] (which I haven't read myself) has more (fictional) details about what can go wrong.
I highly recommend this book. It covers an entirely plausible scenario that could happen in real world, which is very interesting to read, but also - very scary to think about.
> If you can make power usage unexpectedly go up by more than ~10% within a minute, most power grids will fail.
Wouldn't that behave identically to a sudden loss of generation? The power grids I know of have schemes to deal with that, by automatically shedding large blocks of load in several stages.
Yes - but IMO, as soon as you've shed any significant amount of load, you've failed.
If just 10% of a nationwide grid is down, there's a good chance the phone network won't work, internet will be down, trains won't run, credit card/payment systems won't work, etc. All those things have primary and backup systems, but somewhere in the chain of dependencies there will be both a primary and backup that have been shed, and the system designers thought "these two data centers are 500 miles apart, so won't fail together".
Fortunately many critical infrastructure system and life safety systems have on-site backup energy generation capability, as long as diesel is available. At least in the USA, our internet, payment, and phone systems have all withstood significant medium-term regional power outages.
A major infrastructure cyberattack seems effective either as an opening salvo in a traditional military war or to multiply the chaos after a terrorist attack. Taking out the power grid would worsen traffic congestion and create increased demand on emergency services in the short term, and have major economic impacts and high visibility in the long term.
This is an amazing lack of security best practices. To me, this screams outsourced. Given how many people hate Tesla, they need to be taking this seriously. This truly blows me away. This is "people should be fired" levels of organizational incompetence. There's no way some of these issues haven't already been noticed and put in the issue tracker. They're just not taking it seriously. It reminds me of Boeing to be perfectly honest.
And not just fired for this, but because they are incompetent enough to do something like this in the first place. And that needs to be at however high of a level it was that approved the security plan for this product.
Did they even try to submit these issues to Tesla? They have a bug bounty program and have been reasonably good about patching issues in vehicle software. If not, this is pretty irresponsible disclosure.
"Responsible disclosure" is an invention of vendors who want you conforming to their policies and timelines (and more).
Tesla is also "good" at disabling aspects of people's property (like ethernet ports, or ability to receive future firmware updates) when they dislike what people find "wrong" or otherwise in Tesla software.
You can do responsible disclosure without following a vendor's timeline. Project Zero comes to mind. It's not like the only options are either dropping 0 days or doing exactly as the vendor says. Simply give a date.
At the end of the day the point of security research is to make things more secure. Oftentimes that is best achieved by working with the vendor.
I think this comment highlights a lack of understanding what responsible disclosure is about. It's there to reach the best possible tradeoffs to protect consumers and force a quick turnaround with fixes. Just publishing vulns, which the vendor might not even see or learn about(!), will not help in getting things improved and puts consumers knowingly at risk at scale.
Perhaps you should come up with your own term to describe such disclosures instead of co-opting the well established one that means something else entirely?
I understand perfectly well what it is. But see my sibling comments, that reflect an agreement with the concept of "coordinated disclosure", rather than "responsible disclosure", which gives an implication that I am being irresponsible if I do not work to the vendor's needs and priorities.
"coordinated disclosure" is a newly invented term that I've never seen used anywhere in industry. Please don't simply invent new vocabulary and then define it as the standard.
> "Responsible disclosure" is an invention of vendors who want you conforming to their policies and timeline
No it’s not. It’s an invention of ethical hackers and academic security researchers who are mature enough to realize that dropping 0-days to the public is worse for society than giving the vendor a reasonable opportunity to fix it.
“Economics of information security” is a field that publishes some studies about vulnerability disclosure strategies and maximizing the social good.
> That may feel good to say, but as someone whose job it was to find these kinds of bugs in software from companies ranging from tiny startups to financial exchanges to major tech vendors, this is a kind of carelessness shared by virtually everyone shipping any kind of software anywhere.
> That said, the term "responsible disclosure" is Orwellian, and you should very much avoid using it.
> It's coercive. It redefines language to make any handling of vulnerabilities not condoned by the vendors who shipped those vulnerabilities "irresponsible", despite the fact that third parties who discover vulnerabilities have no formal duty to cooperate with those vendors whatsoever.
> The better term is "coordinated disclosure". But uncoordinated disclosure is not intrinsically irresponsible. For instance: if you know there's an exploit in the wild for something, perhaps go ahead and tweet the vulnerability without notice!
> That said, the term "responsible disclosure" is Orwellian, and you should very much avoid using it.
It seems you disapprove of the phrase "responsible disclosure" because it's ambiguous and can be used as a cudgel. That's no different than the term "Orwellian", which is ambiguous and can be used as a cudgel.
All people are saying is that it's better to give the vendor a heads up before releasing an exploit. Maybe they have something to tell you that might cause you to delay; maybe not. Surely responsible disclosure implies to most people some amount of due diligence in choosing a timeline for disclosure. If you don't give a vendor any opportunity to discuss, there's been no such due diligence on your part.
> This might also be ambiguous, but we all understand the genocidal connotations.
Ah yes, because using the term "responsible disclosure" is very clearly analogous to being a white supremacist.
There's no need to be so ridiculously dramatic -- "responsible disclosure" was objectively the term-du-jour used by the infosec community until fairly recently. And it's perfectly understandable why the community changed their mind (vendors started co-opting the phrase as a method of shaming researchers for doing the right thing), but you are shooting yourself in the foot by being so aggressive about it. Why not just say "we used to use that term, but we use this other term now because of the following reasons"?
You've got the order of events wrong here. The term "responsible disclosure" originated from the legal teams of vendors around 2002, it was never co-opted by vendors.
I think the 14 words comparison is apt. Without context there's nothing obviously wrong with the 14 words, just like there's nothing obviously wrong with "responsible disclosure".
tptacek is just trying to speak from a position of authority, there is no substance to his argument. The position is that it’s a term used by vendors... therefore bad? I’ve already pointed out there is an entire academic subfield to infosec that uses the term that has no financial incentive like a vendor.
Releasing anything that can be used to harm the public has ethical implications, period. There is no grey area there. Responsible disclosure is entirely about minimizing impact.
People that don’t like the term don’t like it because they don’t like the idea that there are ethical implications for releasing information.
Do you think there is an ethical gray area to releasing a method to the public to gain access to the secure side of an airport terminal without going through security so guns/bombs can be brought in undetected? What if that method is a software vulnerability in the door locks they use?
> The better term is "coordinated disclosure". But uncoordinated disclosure is not intrinsically irresponsible. For instance: if you know there's an exploit in the wild for something, perhaps go ahead and tweet the vulnerability without notice!
FFS, that’s why the term is “responsible disclosure” and not “coordinated disclosure”. Responsible disclosure is the entire process of determining when and how to inform vendors and/or major users or if to inform them at all. Responsible disclosure is entirely about doing disclosure in the least harming way possible to the public.
I have followed this shift in vocabulary -- now the term for not dropping a 0day is "coordinated disclosure" and usually implies that the researcher has final say on the disclosure timeline. But what I don't understand is why so many folks in infosec pretend as though "responsible disclosure" has always been a dirty word -- from memory, it was the primary term used by the infosec community until only a few years ago.
> "Responsible disclosure" is an invention of vendors who want you conforming to their policies and timelines (and more).
Ridiculous. Please don't put out this kind of nonsense. Responsible disclosure is to protect the users of software. Supply chains cannot and do not respond instantaneously. You cannot fix a security problem within microseconds. There MUST be some delay between disclosure to the software provider and the actual release. Otherwise you're just endangering the public. How much delay is certainly a point of discussion, but no one is saying "no delay".
This doesn't really even seem like much of a disclosure to be had. The system is mostly open without authentication, and what little auth exists is behind a trivial-to-break password.
I mean hell, the main "hack" is looking at the API calls.
This is what corporations claim, an example that makes you think about the "irresponsibility" of disclosure is the Intel security bugs, Intel sits quiet for 1 year (and more if it's bribe would have worked) and customers are tricked to buy insecure products. As a possible future customer you would like to know that something is insecure before you buy it , keeping it hidden will just increase the number of affected people.
You're worried about future customers, but what about Tesla's existing customers? For their sake, researchers should at least give the company a chance to respond and fix the issue before public disclosure.
I agree that a year is too long. I think 30 days is about right, but it depends on how fast the product is selling, how many new customers might be harmed during the 30-day period vs. how many existing customers there are.
How many customers do you think have already purchased a PowerWall, vs. how many would have purchased in the next 30 days?
I would agree if and only if is a super hard to find issue, but if any curious teenager can find and abuse this then keeping is secret is making more harm.
If your car had a bug where it could be controlled remotely by any teenager with a laptop and a program do you want to know and stop using the car or block it's internet connection or prefer Tesla has it's 90 days to find a fix.
Anyway let's see what harm comes from this being revealed and maybe I will change my mind.
This is not a bug. This is irresponsible behaviour on the side of Tesla, providing hardware that is so open to abuse. With potential effects to the grid as well.
If it is documented in the manual[0][1] is it a "bug?" I think the design is poor (bad default WiFi password, WiFi always on, inverting sensors with no sanity checks) but this is how it was designed to work, it is officially documented as doing so.
I guess it boils down to a definition of the term "bug." But I feel like a lot of the knee-jerk responses (and voting) didn't read the article carefully enough.
It is a core design weakness rather than a "bug" (implying an error) in an actual good design. I think the distinction is subtle but it is there.
Not having any limits on that invert setting is definitely a security/safety bug.
The password is a level of incompetence beyond what you'd call a bug. It's clearly following a design decision that was imposed by someone with zero security experience.
I would argue that "building, and leaving open" Wifi connectivity that is not even remotely obfuscated is proof in itself of "failure to consider security aspects".
Well, to be fair and comparing to not having a password at all - as they put those passwords in place they have somehow tried to protect it. Or make it look protected.
But I agree that it indicates that they haven't gave any consideration to it, and in 2019 that doesn't count as trying anymore. At least because we're talking about a well-established hardware and software vendor, not my grandma.
This is unintended behavior in the same way that a locksmith leaving a copy of a key under your doormat after installing a new lock is unintended behavior. They didn't intend to cause a security issue, but they also didn't care enough to make sure that what they were doing was safe.
It's not clear that most of this is even "unintended behavior" rather than just abuse. It's like calling it a "hack" to fill water balloons with electrically conductive fluid and throw them at a substation. The people responsible for stopping you from doing that are the police, not the makers of water balloons.
The worst thing they did was the default password.
Complete negligence in security is hardly the kind of thing people think about as "bugs". The larger problem is that Tesla has put out software without any sensible considerations for security - that the result behaves badly can't really be called unintended behaviour, unless you are allowed to design software by pipe dream.
> Anyone interested in powerwall vulnerabilities would find them trivially.
What about those who become invested in powerwall vulnerabilities because they read an article on HN that showed how easy it is?
> When you disclose such things quietly with the vendor, it mostly just saves their face.
I don't see how. They still get put on blast for ever being vulnerable in the first place. If you're goal is to get them in more trouble at the expense of their consumers, then that seems like pretty irresponsible disclosure to me.
> Not everyone values saving Tesla's face.
And not everyone cares one way or another about Tesla. This is about responsible disclosure. It's about protecting the consumers. It has nothing to do with the vendor.
It can also give you a five- or six-figure payout and prevent unsuspecting customers of being hit by a disrupting and damaging attack, causing unnecessary economic damage. IMHO, choosing the priority of embarrassing someone rather than give them a 24-hour window of notice to avoid this would be a shitty move.
This should be short-term trivially fixable by pushing a password update that sets unique passwords based on a SN -> password mapping stored in a DB at the vendor. You don't need more than 24 hours to do that.
What is the bug exactly? Its not like there is some complex sequence to trigger any of this shit, its just sitting there because it's a terrible unsafe design. This is more of a critique of design than any zero day bug post. This is all in the goddamn manual.
There is no such thing as irresponsible disclosure. You are allowed and encouraged to publish your security research WHENEVER YOU FEEL LIKE.
The term “responsible disclosure” is a shady tactic to frame publishing immediately and in full to those affected by the vulnerability as irresponsible. It is not.
> The term “responsible disclosure” is a shady tactic to frame publishing immediately and in full to those affected by the vulnerability as irresponsible. It is not.
What? Have you even tried to understand what responsible disclosure is?
It is about giving the vendor a chance to protect their consumers before teaching the public how to exploit a vulnerability.
The vendor still gets put on blast when the vulnerability is eventually disclosed. But at least there is less risk of the vulnerability being exploited in the mean time.
And if the vendor doesn't fix the vulnerability in a reasonable time frame, they get extra blast. We've seen this happen many times. The recent Valve and Intel incidents stand out to me as good examples.
The vendor’s existence and actions are irrelevant to whether or not it’s okay to publish your research at any time. It is never irresponsible to disclose one’s research in public, in full. Vendors don’t factor in to it at all.
Their customers do, though, and that's the whole point. Without giving people a short, reasonable time to patch, you're part of the problem, not part of the solution. For what it's worth, you are also coming across as something of a zealot, rather than a reasonable person who sees the world in shades of grey.
The burden of proof lies with the person who says publishing research and making it immediately available to those it directly affects is irresponsible. It is not.
The side effect that it is available to attackers is also irrelevant to the determination of whether or not it’s irresponsible; it also becomes available to the vendor who shipped it.
This notion of “you have to give the vendor time to patch xor you are irresponsible” is itself bogus. The vendor releasing the patch (prior to publication of the paper) is just as much a drop of the 0day as publishing the paper is.
> The burden of proof lies with the person who says publishing research and making it immediately available to those it directly affects is irresponsible. It is not.
Burden of proof? This isn't a court case. The overwhelming majority of users will never see a vulnerability write-up, do not have the education needed to understand such a writeup and - even if they saw the writeup and understood it - lack the tools and information to fix a vulnerability in proprietary software.
The vendor only factors in because they are the only one who can fix the issue.
A disclosure like this only helps hackers and the few consumers who are capable of hacking their own product. It hurts everyone else.
> The vendor releasing the patch (prior to publication of the paper) is just as much a drop of the 0day as publishing the paper is.
Who here is arguing against disclosure? We're arguing for responsible disclosure. Once the vendor distributes patch, consumers can actually do something to protect themselves. At that point, you can put the vendor on blast all you want without harming the consumers.
Is there an alternative to the powerwall that is functionally similar, in terms of power density and utility, but doesn't have things like a network stack and other wifi / IoT / "smart" features ?
We are just about to install lithium battery backups in our home and, of course, the Tesla powerwalls are an obvious choice, but I don't look forward to having to disable/reenable the network connectivity on them, manually, in some cat-and-mouse with Tesla for updates or whatever...
In fact, I'll bet that most (all ?) of the updates that Tesla has to send the powerwalls are for the update system, and accompanying network stack, itself ...
I got this instead of a powerwall because
1) price per kwh
2) tesla will not let you install your own powerwall, you need to go through an installer and attract all those associated costs (which are significant)
3) better integration with a mixed solar equipment install and pv array
4) powerwall does not support an offgrid installation, or at least did not when I looked into it
5) has canbus to a BMS, you're never obligated to connect to the internet
I'm amazed Tesla went ahead and used easily guessable and unchangeable passwords. It's just such a trivial and obvious issue that it seems strange Tesla engineers let this happen.
I wonder if this was an off-the-shelf solution they're using here, or was developed by external contractors, for lack of a better explanation.
I have a couple of PW2s installed, connected via ethernet only (isolated on its own VLAN, though Tesla is total garbage about basics like "what firewall rules are needed"), no cellular here either. But the TEG-$(SERIAL){3} network has always been right there anyway, which is just really lazy design. I happen to be physically far enough away from anyone else that it's very unlikely to be a security issue in practice, but it's still unnecessarily polluting WiFi even beyond that. This has been noticeable for a long time too, it's not hard to find threads going back a long ways with people asking how to disable the gateway WiFi, ie., ( https://teslamotorsclub.com/tmc/threads/disable-gateway-wifi... ), or people speculating about the obvious vuln/interference implications. From that thread back in June:
>It would be nice to find a way to turn off the TBG's WiFi hotspot. I already have too many WiFi hotspots in my area for my taste. TBG's broadcasting WiFi is just a exploit waiting to happen.
The whole thing is genuinely perplexing. Dependence on WiFi, while regrettable, I guess can sometimes make sense from a "user friendly" perspective for a lot of typical consumer installed gear. But the PowerWalls are absolutely not consumer installable, nor obviously in any way inherently wireless. They represent serious electrical infrastructure, and require professional installation with a lot of run cable. Adding in some shielded cat 5 or whatever along with that is frankly trivial for multi-thousand/ten-thousand dollar professional projects, even when not dealing with people who can do a drop themselves.
Minimizing attack area is really trivial security, and here it's got other bonuses like just being more reliable. The entire IOT space is full of shit of course, but it seems much stranger in this instance to me than something like lightbulbs. And why even have passwords at all for access versus using keys? None of this is supposed to be generally accessible anyway. It's not like this would cost Tesla anything extra.
Edit to add: HOSTNAME INCLUDES THE FULL SERIAL. I thought I'd take a second look at this and just pulled up the client info for the PW2 Gateway on my network, and hostname is 11XXXXX-00-J--S$(SERIAL). So no local physical access is required, the gateway itself just broadcasts the whole serial, which in light of this is an, interesting, decision. I can confirm that using the hostname with an added S at the front (so on mine serial was T[...], I used the password ST[...]) I was able to connect to the WiFi spot, and in turn to the management page described. Incredible.
Incidentally DPI is also kind of wacky now that I look, just running a simple Suricata setup but connections are all over the place (why is YouTube showing up?). About 4.5 GB down and 9 GB up, down I assume would be updates of some kind, up monitoring data though that seems like a significant amount for what should generally be text. I haven't had it that long, must be kind of chatty.
>Edit to add: HOSTNAME INCLUDES THE FULL SERIAL. I thought I'd take a second look at this and just pulled up the client info for the PW2 Gateway on my network, and hostname is 11XXXXX-00-J--S$(SERIAL). So no local physical access it required, the gateway itself just broadcasts the whole serial, which in light of this is an, interesting, decision. I can confirm that using the hostname with an added S at the front (so on mine serial was T[...], I used the password ST[...]) I was able to connect to the WiFi spot, and in turn to the management page described. Incredible.
this is staggeringly irresponsible design! thanks for the confirmation
I guess in practice it doesn't really matter much anyway because while my serial doesn't specifically follow the format described in the link it'd still be utterly trivial to brute force. So that they leak the whole thing onto LAN anyway just turns it into even more of a joke, but I guess would rarely make a real difference. Some of the HN crowd have security monitoring on their own networks that would detect suspicious hammering patterns, have more device isolation, etc, but I strongly suspect the vast, vast majority of PW installs are pure ISP default setting CPE environments. Using crummy default password routers full of their own holes too!
The criticism of this security vulnerability has been well covered at this point, so I'd just like to point out that this should be trivially solvable in a few hours by changing all passwords from HQ.
It'd be quite irresponsible of this researcher if they didn't give notice beforehand (don't know if they have).
So many people like to nitpick when it comes to Tesla, It reminds me of the Apple critics in the early days of the iphone. They assume Tesla should have the highest standard and be absolutely impeccable with all their products.
Just don't buy it if you don't like it. Let the rest of us enjoy a sustainable future with insanely safe full self-driving electric cars.
this is an exploit that allows an attacker to disrupt the electrical grid - even if it's only to the substation, that effects all of your neighbors that didn't encourage you to blindly support Tesla's endeavour. it's highly irresponsible to have this kind of control behind such a trivial login on something that connects to your power lines, something extremely dangerous. far from a "nitpick"
An increase in embedded generation is the same as a decrease in load to the system. So turning a heater off is akin to “pushing power in to the system”, unless the embedded generation is larger than the load on a feeder, in which case the direction of power flow would be reversed, which may or may not cause problems depending on the protection settings at the substation.
The peak system load and generation on the Western North American grid is 167,000 MW. There is so much inertia there a 10kW power wall isn’t going to do anything.
This is absolutely true, and the blog post itself definitely has a little of that "picking on Tesla" vibe. And I think it's unfair too.
But at the same time, this is pretty genuinely bad security architecture, and it really needs to be exposed and embarrass the company that did it. Tesla does some great stuff. This thing seems a little mailed-in, though.
Reread the bill, and it actually says: “The preprogrammed password is unique to each device manufactured.” If the default is based on the serial number, I guess it’s “unique” under the letter, but certainly not the spirit.
Link to bill: https://leginfo.legislature.ca.gov/faces/billTextClient.xhtm...