> All it takes is one tiny subpopulation with more than 2 kids per woman, and you get exponential growth.

There's an essential factor missing from this assumption: sustained birthrate above replacement.

I realise you probably intended to mean that, but it's an important distinction because there are at least a few other important factors at play.