It's an issue with the zoning laws. Back when it became illegal to ban black people from your neighborhood the next best thing was to ban the kind of housing that black people could afford. So the only place where it's legal to put the kind of housing that addresses the problem is in the city centers which were already established and therefore out of scope for the zoning restrictions. (more on this: https://www.youtube.com/watch?v=SfsCniN7Nsc)