I don't think you are correct about speed limits. Most limited access roads can probably support higher speeds (Except when there is a lot of traffic entering and exiting), but the majority of 2 lane roads don't have the visibility to support much higher speeds (More developed roads probably do, but they are small fraction of the total mileage).
Roads that see much pedestrian use certainly don't need higher limits (they don't necessarily need lower limits either, just some better sharing system).
I'm in Canada, so it's possible that my experience differs. I will edit the post to clarify that I'm talking about arterial roads and highways though, not residential streets. Most main non-highway roads here still have limits of 50km/h (30mph), with the average speed ranging around 20% higher than that. Most highways have limits of 80km/h (50mph), with average speeds perhaps 10-20% higher, and IMO safe speeds in good conditions often 50% higher.
Roads that see much pedestrian use certainly don't need higher limits (they don't necessarily need lower limits either, just some better sharing system).