I have read the criticism of how the Autopilot miles aren't apples-to-apples comparisons with national averages many times. However, this cherry-picks a single number from the safety report and ignores the other reported statistics. If the explanation for why Autopilot miles were so much safer than non-Autopilot miles is that people turn it off in dangerous situations — and thus equal or greater numbers of crashes were occurring for Autopilot users overall compared to the national average, they were just occurring when Autopilot was off — the crash rate without Autopilot engaged would have to be higher than the national average. Otherwise, where would the crashes go?
However, it isn't. The crash rate with Autopilot off (but with other safety features on) is about 4x better than the national average. And with all safety features turned off, it's still 2x better.
I don't think you can explain away the high safety record of Autopilot by claiming the crashes are concentrated in the non-Autopilot miles, because they aren't. While Autopilot miles are safer than non-Autopilot miles, non-Autopilot miles are no more dangerous than the national average (and in fact are less dangerous).
Autopilot+human is considerably safer than human alone.
Even if what you argue is true, it doesn't follow from this report. Why is the accident rate of Tesla with Autopilot and all safety features off 2x better than the national average? Because there is a difference in the demographics - Tesla drivers are probably younger and more enthusiastic about driving than the average driver.
Now, if you do the same statistics on the same demographics for all non Tesla cars, you could actually get less accidents than Tesla - here are where the hidden accidents went. Again, I don't have the data about this and I don't claim it is true, but without knowing this, you cannot make the conclusion you are making as well.
Otherwise I agree with you - I also believe that Autopilot+human is safer than just human. Unfortunately, the usual way that people interpret these numbers is that Autopilot is safer than human...
I agree that the demographic skew probably accounts for some of the difference. Probably also that Teslas need less maintenance (esp brakes, due to regenerative braking), so are less likely to fail for mechanical reasons — although I don't think most crashes are due to mechanical failure, it should show up to some degree in the stats.
I think the argument that the Autopilot numbers are essentially fake because the true crashes are concentrated in the Autopilot-off scenarios is hard to make a case for though, given the stats on Autopilot-off driving being so comparatively good. You would need incredibly good demographic skew to account for that if the crash rate is concentrated — you don't need to just equal the average after correcting for demographic skew, you need to be considerably worse than it. So while it's not a perfect metric, I would be much more surprised if Autopilot+human was more dangerous than human alone.
I 100% agree with you that this is only an argument for Autopilot+human though. Current Autopilot without humans, at least anecdotally (I have a Model 3 with Autopilot), does not seem safe. However, I think the concern among some that Autopilot is unsafe as it currently is typically operated — i.e. with a human in the loop — is largely contradicted by the evidence.
My personal anecdote is that I feel much less fatigued by driving with Autopilot, especially on longer drives. It's imperfect, but it actually generally helps improve my alertness because I don't have to constantly fiddle cruise control settings based on which car is in front of me or make micro wheel adjustments to stay centered in a lane; I usually take over proactively whenever there looks like a sketchy situation is coming up like multi-lane merges with trucks for example. And when those situations happen, I'm able to stay more alert and focused because I haven't been spending my energy on the simple stuff that Autopilot is good at, so I think I end up being safer overall even when it's disengaged. I notice a pretty large cognitive difference — which was unexpected for me when I first got it, because I thought I probably wouldn't use or like Autopilot, and initially was quite mistrustful of it.
Obviously this is just a personal anecdote, and not data! But what data we have, while imperfect, seems to support that conclusion much more than it supports the opposite.
Expanding upon your personal anecdote: Is there scientific research on this matter? (Measuring alertness/fatigue on non-assisted vs assisted driving) It could be valuable.
Personally, I think driving is nearly always a waste of my time, so I avoid it when possible. Plus, I don't think of myself as a very good driver. Reading your anecdote made me think about how I feel after a long drive vs a long train ride. I cannot put a finger on it, but fatigue from constant required adjustments when driving /might/ be a factor.
More likely: I like how I can spend my free time when riding a train vs driving a car -- which is somewhat limited to passive listening: radio/music/audiobook/podcast/etc.
Don't younger (hence less experienced) drivers generally have more accidents? If this is true, isn't it more evidence that Tesla's safety features are helpful?
I think "younger" here is meant more as "not old." 16 year olds are less safe drivers, yes, but on account of the price they're not going to be a big part of Tesla's demographic.
Since there's no affordable model, and they're a newfangled gadget with strange refueling infrastructure and a giant touchscreen for a console, Tesla owners probably skew toward middle aged. So they'll have fewer drivers in the less safe age ranges at both ends of the spectrum.
That depends on how you define younger. Not many teenagers can afford a Tesla though so in this case younger probably means mid 30s to early 40s. That largely removes very inexperienced drivers and the elderly.
Your last paragraph is the most important one. Autopilot is driver assistance, and it shouldn't be a surprise that it helps. But these results are comparing human + computer vs human, and does not in anyway indicate that the computer alone is better than a human, let alone a human + computer, which should be the benchmark.
I agree these numbers only argue for human+computer vs human, and not computer vs human.
I'm curious why you think the benchmark should be computer vs human, though. Autopilot is very clearly a human+computer system; it states you need to be alert, and it forces you to keep your hands on the wheel and apply wheel torque occasionally to make sure you're actually paying attention. Why would Tesla benchmark something they don't ship (and how could they even do that)? The question for the general public, and for Tesla owners, is whether the current system is safe. It appears to be.
Theae stats are often quoted by Musk and Tesla to suggest that driverless cars are here and safer than human drivers, and the only thing preventing them are regulators. They are never quoted to imply that driver assistance makes driving safer, which i believe they would.
So, one has to compare computer vs human. In fact, more than that. One cannot compare modern technology to one from the previous century. So one must compare computer to the best passive driver assistance that one can develop for humans. So Tesla must compare a driverless solution to their own driver assistance solutions aiding drivers, and not the "average car on the road"
> Theae stats are often quoted by Musk and Tesla to suggest that driverless cars are here and safer than human drivers, and the only thing preventing them are regulators.
That's surprising, I hadn't seen that. Could you link to an example?
Yeah, and in the part of that second link that directly addresses the question:
> *Will the FSD Computer make my car fully autonomous?*
> Not yet. All Tesla cars require active driver supervision and are not autonomous. With the FSD Computer, we expect to achieve a new level of autonomy as we gain billions of miles of experience using our features. The activation and use of these features are dependent on achieving reliability far in excess of human drivers, as well as regulatory approval, which may take longer in some jurisdictions.
That clearly states that there is still a technical challenge to overcome which is prior to any regulatory issues.
> However, it isn't. The crash rate with Autopilot off (but with other safety features on) is about 4x better than the national average. And with all safety features turned off, it's still 2x better.
You still can't figure that out from Tesla's stats. It'd have to be "compared to the same roads in the same conditions". Tesla only knows where its vehicles have been driven, not every vehicle on the road. Let's be honest, this stat is just marketing.
It is not "dishonest." Toyota, AFAIK, does not publish these numbers; comparing to the national average is just the best you can do. Publishing the numbers without any comparison would be silly; what does it mean to know Tesla's accidents per mile if you not only don't know it for any other manufacturer, you also don't even know what the national average is?
And while I couldn't find numbers for Prius specifically, it seems that hybrid cars are actually on average more dangerous than other cars, so I would be surprised if Tesla were not handily besting the Toyota Prius given Tesla's safety record: https://www.thecarconnection.com/news/1022235_hybrid-drivers...
Yes, there may be biases in driver population that make Tesla owners slightly more or less likely to crash. However, I think it is a very large stretch to claim that this would result in the fairly astoundingly different safety numbers.
As for the age of the car: car age is mostly a statistical factor due to safety systems in newer cars. (It is also important in terms of deaths due to safety standards like crumple zones and airbags, but we are talking about a count of accidents, not deaths; if a crumple zone has been used, it is an accident.) Tesla publishes the statistics both with safety features on (4x better than national average), and the numbers for if they have been disabled which is still 2x better.
I think if the claim that the crashes are concentrated in the non-Autopilot miles were true, and that Autopilot+human is more dangerous than human alone, it would be very hard to understand how the crash rate was still 2x better than the national average with safety features disabled and Autopilot off.
> It is not "dishonest." Toyota, AFAIK, does not publish these numbers; comparing to the national average is just the best you can do.
When you know, unequivocally, that you are missing huge swathes of information, and drawing all manner of conclusions and inferences not supported by those statistics, it's not "just doing the best you can", it's "being disingenuous and misleading with numbers".
The real world is not perfect. People have to make the best decisions they can with the data available.
I think it's better to publish the numbers that are available. Tesla can't publish Toyota's numbers, because they don't have them. I don't think they're at fault for comparing against the only benchmark available, and I think it's better to have that comparison than not. Many, many writers have claimed that Autopilot is inherently unsafe and believed it would cause massive numbers of crashes compared to traditional cars. The data shows that not to be the case.
If you make your decisions based on partial information, knowing that large parts of the information is missing, and then pretend that it's not, you are not making the best decision. The best decision must take into account that information is indeed missing.
Really? I think I remember reading that accidents in newer cars are more rare. How does anybody know that? Can we not at least compare to similar aged cars?
The quoted statistics on either side are not helpful here. See:
>> Driving to Safety
>> How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability?
>> Key Findings
>> Autonomous vehicles would have to be driven hundreds of millions of miles and sometimes hundreds of billions of miles to demonstrate their reliability in terms of fatalities and injuries.
>> Under even aggressive testing assumptions, existing fleets would take tens and sometimes hundreds of years to drive these miles — an impossible proposition if the aim is to demonstrate their performance prior to releasing them on the roads for consumer use.
>> Therefore, at least for fatalities and injuries, test-driving alone cannot provide sufficient evidence for demonstrating autonomous vehicle safety.
Note also that Tesla's numbers are reported after several years that Tesla cars with Autopilot have already been driven on public roads. Whatever the numbers say now when Autopilot was first released there was no evidence of it being safer than human-driven cars, only wishfull thinking and marketing concerns.
Please correct for demographics. The average Telsa owner does not include poor people driving beaters with bad brakes, so there's a heck of a lot of self selection going on that is probably skewing the statistics.
However, it isn't. The crash rate with Autopilot off (but with other safety features on) is about 4x better than the national average. And with all safety features turned off, it's still 2x better.
I don't think you can explain away the high safety record of Autopilot by claiming the crashes are concentrated in the non-Autopilot miles, because they aren't. While Autopilot miles are safer than non-Autopilot miles, non-Autopilot miles are no more dangerous than the national average (and in fact are less dangerous).
Autopilot+human is considerably safer than human alone.