I guess it varies from individual to individual. I can be out in 110F weather all day comfortably if I have enough water and a large hat but have very low tolerance for the cold.
Inside the sleeping bag, it will definitely be quite warm!
Even quite mediocre sleeping bags (by modern standards) are positively toasty at 32 degrees. It was common in my group for people to sleep in their underwear in the caves! Our sleeping bags were 0 degree rated, though.
Further: the major source of heat loss in a sleeping bag isn't to the air, it's to the ground. Which is why backpackers carry sleeping mats, either of expanded polystyrene or self-inflating air-mattresses. In both cases, the air is the insulating element.
In primitive conditions, pine boughs or fur skins would be used.
Other than that, wear a cap, and if it's truly cold, boil water, pour it into a Nalgene (or similar) bottle, and slip that into a sock. It'll keep you toasty (often too warm) all night.
