In theory, yes. In practice, we've built our roads, signals, and car interiors, tailored to our specific minds and concepts.

So when we see someone that just started driving perform well under some circumstances, we can good performance under circumstances that are similar to the human mind. The problem that the "fooling neural networks" experiments show is that two things that are similar for humans can be wildly different for a NN that's been trained to recognize them.