I do appreciate the effort you went to for this, but you make a lot of very strong claims that depend on very obscure minutia and unfortunately they end up being false under careful scrutiny.
>But who says my fancy pointers will even necessarily map 1:1 to a linear (and contiguous!) pointer space?
The standard as of C++11 does. std::vector<T> provides the member function T* data() which is required to return a pointer to the first element of a contiguous memory region spanning the entire vector:
>What we see here is folly pretends to support custom allocators, but it cuts corners internally.
No such corners are cut. Folly has full support for custom allocators including fancy pointers and in this case it's giving you a compile time error for what could have been undefined behavior. That's an advantage in my book, I'll take compile time error over hard to debug memory corruption any day of the week.
Fancy pointers to T are required to be implicitly convertible to T* and while the small snippet of code you pasted doesn't make use of this requirement in GCC and clang, MSVC does make use of this requirement and you can see as per the link below that your allocator fails to work:
You'll notice the error is that std::vector is trying to convert your fancy pointer into a raw pointer in order to call _Delete_plain_internal:
>C:/data/msvc/14.28.29910/include\vector(683): error C2893: Failed to specialize function template 'void std::_Delete_plain_internal(_Alloc &,_Alloc::value_type *const ) noexcept'
Other parts of the standard that depend on this are node based containers such as std::list, std::set, and std::allocate_shared also makes use of this requirement.
The fact is that writing a standard conforming allocator that uses fancy pointers is incredibly difficult and error prone, the standard is not clear about the rules with numerous defect reports related to fancy pointers and as of today no compiler actually has full support for it:
The fact that you could have picked any example to prove your point about the standard library and the one you chose to produce took you a very long time to exercise what is a fringe and corner case example of a defective language feature that you do not fully understand is not a particularly convincing argument that the standard library provides the high quality implementations and APIs.
That may seem blunt and harsh, but you're in good company since I myself also don't fully understand it, and anyone who's being honest would also admit that fancy pointers in C++ are an area that quite possibly no one really truly understands.
Before jumping to the conclusion that Facebook got it wrong, they cut corners, and their code is not trustworthy over this, perhaps it might be best to pause a moment; this issue has nothing to do with Facebook and instead is merely a reflection of the sheer complexity of C++ as a language.
> The standard as of C++11 does. std::vector<T> provides the member function T* data()
I actually do believe data() is a defect in the standard. It should be returning 'pointer', not T*. In fact I believe the entire contiguity requirement is a defect, as there's no inherent reason for a vector to require physical contiguity in memory to begin with. Contiguity needs to be with respect to the pointer. Getting to that point is not trivial, though, given so many implementations have assumed raw-pointer-like behavior in the past, so that likely plays a role in why they use raw pointers in places like this one.
That said, you do have a point here, in that the standard requires physical contiguity in memory for std::vector, so this isn't a bug as far as memory corruption goes. In my rush to write up an example, I forgot about this with respect to std::vector, and I just assumed data() would return 'pointer' as would be common sense.
> Fancy pointers to T are required to be implicitly convertible to T*
Do you have a link? I'm failing to see this in [1] or [2]... is it cited elsewhere? Its existence would seem to render the addition of std::pointer_traits<Ptr>::to_address() rather redundant.
> Folly is giving you a compile time error for what would have potentially been undefined behavior.
folly is not giving me that error though... my own code is. By not defining an implicit conversion to a raw pointer. If I had added that, which many would, then I wouldn't have gotten the error.
> You'll notice the error is that std::vector is trying to convert your fancy pointer into a raw pointer in order to call _Delete_plain_internal:
We got different behavior because you're using the debug runtime (/MDd) and I wasn't (/MD). I tried it without and didn't get that. So yes, MSVC also gives an error with the debug runtime.
> the one you chose to produce took you a very long time to exercise
No, this is twisting what happened. It took an hour to write, not to "exercise" the 'corner case'. And it takes a long time to write only because C++ is so extremely verbose and it takes almost 100 lines of code just to implement a simple example around a trivial pointer wrapper. Practically everything in the example was basic boilerplate. And even then I later noticed I still missed other boilerplate like the < and > operators.
> The fact that you could have picked any example to prove your point about the standard library [...] to exercise what is a fringe and corner case
Oh come on. I literally said in my comment "they're often (read: not always) far more flexible and handle far more edge cases than open-source libraries do." You cherry-pick some of the absolute best C++ libraries out there as if they're somehow representative of the landscape, then force me to whip up a counterexample for you on the spot as if I have one lying around at my fingertips for every library. And when I nevertheless try to find something to give you an example of like you asked, you complain that it's... a corner case? Didn't I say that it's an edge case to begin with? And isn't this doubly ironic when you yourself cherrypicked libraries that were very much "edge" cases to begin with in terms of their high quality?
> fancy pointers in C++ are an area that quite possibly no one really truly understands.
This is a weird way to put it. This isn't something where C++ is just too complex for mortals to comprehend; it's something where the standard itself has shortcomings (like the ones you yourself linked to). The standard needs to simultaneously (a) provide some kind of generality and usefulness, while (b) attempt to address past issues in a mostly backwards-compatible way. Which is intrinsically hard because in the past it has made assumptions that probably shouldn't have in hindsight. All of which I'm more than happy to acknowledge; I never claimed the standard is flawless or that somehow easy to improve it when it might break a ton of old code.
What you need to realize about fancy pointers in particular is that part of the very reason they're under-utilized is their poor support, not because they're somehow fundamentally a "fringe" concept for people to want. (Unless you think nonstandard allocators are weird altogether, in which case I'm talking to the wrong person.) The commonly-cited use cases (like shared memory) are far more obscure than some fairly normal things you can do with them. For example, they're invaluable when you're debugging custom allocators; you can put things like bounds-checking into them to make sure the rest of your allocator is correct (in fact I'm pretty sure I was doing exactly this not too long ago). But to be able to do anything with them you need a container that is flexible and careful enough to not treat them interchangeably raw pointers.
But this is digging too much into fancy pointers and missing what I was trying to say, and it's making me waste hours more on this than I ever intended to. I was saying, in general, with things that require more flexibility than the obvious implementation would imply, I tend to find better support among standard libraries than third-party ones. This statement obviously has nothing to do with Facebook or folly in particular; you just picked 3-4 extremely high-quality libraries and made me fit what I was saying into a mold that's already a cherrypicked outlier, so I tried to come up with something for you on the spot to get my point across. Whether you think it was a good example or not, it's very much missing a point I was making about 3rd-party libraries in general. We can literally even assume those 4 libraries are flawless, and even then it would hardly even matter for a discussion that's about the ISO standard and its impact on the average C++ library.
P.S. There was a typo in my example for 'rebind'; you need to put U instead of T. You'll need to fix that and other uninteresting stuff (like operator<) to address errors with other containers.
Basically I ask you to give examples to showcase how the standard library handles flexible edge cases with high quality, and when I point out how the example you gave is fundamentally flawed your counter argument is that for your own example, and you could have picked anything: the standard is defective and goes against common sense, it takes 100 lines of code and an hour to implement a trivial example to showcase how flexible it is, it has made assumptions that it probably shouldn't have in hindsight, and a host of other reasons that basically showcase that the standard library isn't nearly as flexible or high quality as you made it out to be.
It was your example to give and it turns out that just providing a basic example requires all this complexity, exposes all these defects, isn't standard compliant and not portable across compilers.
You are certainly welcome to your opinion and I doubt either of us are going to convince one another at this point... but I am fairly confident most sensible people would not look at the example you chose to showcase and think "Wow, what a flexible and powerful API the standard library provides, very high quality." They will come away thinking that your example is everything wrong with C++; it's convoluted, error prone, and incredibly fragile.
>But who says my fancy pointers will even necessarily map 1:1 to a linear (and contiguous!) pointer space?
The standard as of C++11 does. std::vector<T> provides the member function T* data() which is required to return a pointer to the first element of a contiguous memory region spanning the entire vector:
https://en.cppreference.com/w/cpp/container/vector/data
Note that it specifically returns a T* rather than a my_pointer<T>, or what is referred to in standardese as a "fancy pointer":
https://en.cppreference.com/w/cpp/named_req/Allocator#Fancy_...
>What we see here is folly pretends to support custom allocators, but it cuts corners internally.
No such corners are cut. Folly has full support for custom allocators including fancy pointers and in this case it's giving you a compile time error for what could have been undefined behavior. That's an advantage in my book, I'll take compile time error over hard to debug memory corruption any day of the week.
Fancy pointers to T are required to be implicitly convertible to T* and while the small snippet of code you pasted doesn't make use of this requirement in GCC and clang, MSVC does make use of this requirement and you can see as per the link below that your allocator fails to work:
https://godbolt.org/z/7KrE41
You'll notice the error is that std::vector is trying to convert your fancy pointer into a raw pointer in order to call _Delete_plain_internal:
>C:/data/msvc/14.28.29910/include\vector(683): error C2893: Failed to specialize function template 'void std::_Delete_plain_internal(_Alloc &,_Alloc::value_type *const ) noexcept'
Other parts of the standard that depend on this are node based containers such as std::list, std::set, and std::allocate_shared also makes use of this requirement.
The fact is that writing a standard conforming allocator that uses fancy pointers is incredibly difficult and error prone, the standard is not clear about the rules with numerous defect reports related to fancy pointers and as of today no compiler actually has full support for it:
https://quuxplusone.github.io/draft/fancy-pointers.html
The fact that you could have picked any example to prove your point about the standard library and the one you chose to produce took you a very long time to exercise what is a fringe and corner case example of a defective language feature that you do not fully understand is not a particularly convincing argument that the standard library provides the high quality implementations and APIs.
That may seem blunt and harsh, but you're in good company since I myself also don't fully understand it, and anyone who's being honest would also admit that fancy pointers in C++ are an area that quite possibly no one really truly understands.
Before jumping to the conclusion that Facebook got it wrong, they cut corners, and their code is not trustworthy over this, perhaps it might be best to pause a moment; this issue has nothing to do with Facebook and instead is merely a reflection of the sheer complexity of C++ as a language.