A lot of criticism about intelligence tests are ideologically driven, because underprivileged groups score low, and in many regions that have suffered the most from colonialism, underprivileged groups score low. I think that what we will find is that being underprivileged affects intelligence quite strongly.
Sephardic Jews were at the center of worldwide Jewish culture. Their cultural contributions and success in society would suggest that as a population they were probably quite intelligent during their heyday. Today, Ashkenazi Jews are known to have a high average group IQ, higher than current day Sephardic Jews.
I suspect that ideological knee-jerk fear driven suppression of IQ research on underprivileged groups is keeping our society from realizing interventions which can help groups by driving up their average IQ scores. We have historical evidence from the US that underprivileged groups with below average IQs can be changed.