I sure hope they do have a regression suite with thousands of real world sensor-data samples in some normalized format made amenable to unit testing. Assuming such a normalized format is possible to generate, I expect these samples to be open-sourced and standardized. That would give the NTSB a systematized certification procedure: just run the new version of the software on all the known test cases before allowing it to be released.

Still, I suspect that may not be enough - a new weird road condition or construction site can be created at any time. This is why I believe self-driving companies would be better off spending their budget upgrading and certifying specific routes as safe for automation. Certified routes could be subject to special construction protocols and regular road quality audits that ensure automated cars won't run into a non-automatable condition on those routes. It's also easy to verify and confirm that a car works on all certified routes, rather than trying to test the entire North American road network.