The author explicitly states that they believe that Tesla is correct to only include miles driven with autopilot enabled, specifically they say:
"I agree with Tesla’s methodology on Autopilot mileage because the road conditions under which a partial autonomy system is rated for operation (highways, clear lane markings, etc) are systematically different from manually-driven miles."
But if that were the case you must also only include crash data from manually driven cars in those same circumstances. But the majority of automobile accidents do not occur on freeways -- which are the most autopilot friendly roads being driven.
We can't easily work out that traffic split, but we can do something else that should be just as effective:
Compare accidents per million miles driven in autopilot enabled cars, to accidents per million miles driven in regular vehicles. If autopilot does meaningfully improve driving safety, we should expect that the average for autopilot enabled cars to to be lower than other cars by a statistically significant margin.
If we want to get specific per [1] more than 50% of accidents occur at intersections, given Tesla's sample will dramatically over-represent intersectional crashes we could go extreme and say that their statistics should be immediately less than half the crash rate of non-autopilot vehicles. (this is an exaggeration, but given they're not being honest I don't really care)
Personally I'd also control for class of cars we're comparing -- I suspect the stats for high end cars are different from low are different from sports cars, etc.
No, she is saying you shouldn’t just count the miles where autopilot is turned on. Her reasoning is that you can’t compare autopilot-on miles to regular cars because there is no comparable data for regular cars.
That might be acceptable if you compared Tesla autopilot capable cars to other luxury cars, where the only significant difference in safety is autopilot.
But that wouldn’t account for things like: maybe certain Tesla safety features save a lot of lives but get canceled out by a very dangerous autopilot.
> That might be acceptable if you compared Tesla autopilot capable cars to other luxury cars, where the only significant difference in safety is autopilot.
This is the comparison they should have done. Comparing with just the national average includes all the poorly maintained 20-year-old beaters being driven by teenagers.
"I agree with Tesla’s methodology on Autopilot mileage because the road conditions under which a partial autonomy system is rated for operation (highways, clear lane markings, etc) are systematically different from manually-driven miles."
But if that were the case you must also only include crash data from manually driven cars in those same circumstances. But the majority of automobile accidents do not occur on freeways -- which are the most autopilot friendly roads being driven.
We can't easily work out that traffic split, but we can do something else that should be just as effective:
Compare accidents per million miles driven in autopilot enabled cars, to accidents per million miles driven in regular vehicles. If autopilot does meaningfully improve driving safety, we should expect that the average for autopilot enabled cars to to be lower than other cars by a statistically significant margin.
If we want to get specific per [1] more than 50% of accidents occur at intersections, given Tesla's sample will dramatically over-represent intersectional crashes we could go extreme and say that their statistics should be immediately less than half the crash rate of non-autopilot vehicles. (this is an exaggeration, but given they're not being honest I don't really care)
Personally I'd also control for class of cars we're comparing -- I suspect the stats for high end cars are different from low are different from sports cars, etc.
[1] https://www.fhwa.dot.gov/research/topics/safety/intersection...