"Bringing in Boost" doesn't cost you much at all in terms of compile time. Using some of the advanced template meta-programming stuff in it may, but basic things like the legacy compiler and OS support you mention are largely using the C preprocessor and take very little time. In other words, give it a try--I'm sure you won't have half an hour compile times just by using Folly.

If you want to compile C++ (or C) really fast, look at icecc: https://github.com/icecc/icecream