In order for there to be a major accident, there not only has to be a mistake, it has to be a mistake with a major consequence, and it has to be that nobody else there avoided it either.
In general a car that doesn't see a pedestrian isn't a major accident unless the pedestrian also doesn't see the car. If not for that there would be a whole lot more fatalities with human drivers as well. A self-driving car obviously shouldn't do that, but neither should a human, and they do. Which makes it less hard for the computer to do it at least as well.
You can also expect the computers to improve over time by learning from each others' mistakes, which human drivers generally don't.
While it is the case that it usually takes two people to cause an accident, there are plenty of scenarios where there is only one person who is capable of taking the action necessary to avoid it. For example, an elderly person in the crosswalk is not going to be capable of getting out of the way of a speeding car.
It's already the case that pedestrian/car collisions are usually written off as the pedestrian's fault until proven otherwise--the Uber self-driving car that killed a pedestrian is a great example of that in action.
And the same is true of human drivers, is my point. You don't just need the mistake, you need all the other things to go wrong too. The pedestrian entered the crosswalk even though a car was coming, instead of waiting to make sure the car stops first (as pedestrians often do). The car failed to identify something. The thing it failed to identify was a pedestrian and not a mailbox or a trashcan. The pedestrian was elderly and couldn't jump out of the way fast enough etc.
It all has to go wrong at once. Mistakes are common but most mistakes aren't fatal.
> Mistakes are common but most mistakes aren't fatal.
And by choosing to drive a car, you are dramatically raising the odds that any mistake you make is fatal (not necessarily to you).
While my personal experience on the matter is thankfully limited, I strongly suspect that the vast majority of vehicular-pedestrian accidents are mostly (or entirely) the fault of the driver, where the pedestrian started an action where it was safe to do so and the driver rapidly caused it to become unsafe before the pedestrian could react.
In general a car that doesn't see a pedestrian isn't a major accident unless the pedestrian also doesn't see the car. If not for that there would be a whole lot more fatalities with human drivers as well. A self-driving car obviously shouldn't do that, but neither should a human, and they do. Which makes it less hard for the computer to do it at least as well.
You can also expect the computers to improve over time by learning from each others' mistakes, which human drivers generally don't.