There have been cases where the driver attempted to switch to neutral and the car would not shift. Also, there is often not enough time to shift to neutral because the majority of instances of Toyota unintended acceleration take place in parking lots and other confined spaces, and the car hits something before the driver can react.
Here is a link to a video published yesterday with dashcam video of many sudden acceleration incidents.
https://www.youtube.com/watch?v=3oAshG36dNI
These are mainly in Korea, where dashcams are the norm, and likely to be KIA or Hyundai vehicles (they also have big software problems, apparently). Imagine driving in some of these instances of close quarters. Would you be able to shift into neutral to stop in time?
Also, Dr. Antony Anderson, who closely follows this technical issue on his blog, recently published a paper in IEEE Access about how software can be fooled by mechanical glitches such as intermittent connections:
Separately, speaking to your point about shifting to neutral, Dr. Anderson has noted that it is unreasonably risky to design a safety-critical system with the expectation that operator responses such as shifting to neutral can provide an effective failsafe. Would any of you here actually design such a system?