This implies that "fitness" of a society is correlated positively to fertility rates, which for a species which is not exactly on the endangered list, and which is capable of producing results that can be measured on other scales than just warm bodies, seems to be a dubious idea.

Your argument would imply that the 10 "fittest" countries in the world are: Niger, Guinea-Bissau, Afghanistan, Burundi, Liberia, Democratic Republic of the Congo, East Timor, Mali, Sierra Leone, and Uganda.