Prisoners is another population where we would find pervasive brain damage if we scanned them.[0] In time I believe we will find that a ton of destructive/aggressive behavior is due to physical issues in the brain.
When we learned that lead poisoning was bad for you and increased aggression, we didn't kill everybody with lead poisoning. We changed our society to reduce lead poisoning, by getting rid of leaded gasoline, lead paint, etc.
[0] https://link.springer.com/article/10.1007/s00406-013-0403-6