But if a new update changes how the car drives, wouldn't the data that would've been captured by the updated car be different than what the old outdated car recorded?
For example if the new update makes the car more aggressive, then other real drivers might be more careful, slow down more, etc compared to the original runs?
As long as there are human drivers, Google will need to continue updating software. We'll find exploitable behavior (e.g. how to make the self driving car yield) just like we find weaknesses in their search algorithms, and they'll need to adapt.
> I'd assume the more severe the change the more they'd want to test in the real world.
I'm sorry but the important point is: how are we going to agree what needs more testing, and what can be updated without testing? If we let those big companies decide about those issues, then I'm afraid we will soon see another scandal like VW, except possibly with deadly consequences.
I can already predict the reasoning of those companies: last quarter our cars were safer than average, so we can afford some failures now.
For example if the new update makes the car more aggressive, then other real drivers might be more careful, slow down more, etc compared to the original runs?