That is less about header files, and more about how machine code works.
If you want to have some abstract type where you don't let people know anything about the innards, but you do have an explicit interface which enumerates what you can do with it, then yes - you can only really pass around pointers to these things and people outside your abstraction can only pass references not values.
If you want people to be able to pass your abstract type by value (among other things), then either you need to let them know how big the thing is (an implementation detail) or you have to expose the copy implementation in such a way that it could be inlined (more implementation details).
Sometimes, the "pure abstraction" approach is best where you only ever deal with pointers to things, and other times the "let's pretend that people do the right thing" approach is best. I don't see this as a header file thing though.
I disagree with you on this. In another language with explicit public / private separation, the compiler can have access to the internal layout of a type (and thus optimise on it) without letting the developer mess around with it directly. I am assuming static compilation of course. Across a dynamic boundary, I would expect this compiler to behave like a normal C compiler and not use that layout.
In a header file, the information for the compiler and the user are the exact same which means you can't reduce your public interface without straight up hiding more of yourself.
Personally, I'm happy to just let a Sufficiently Advanced Compiler do link time optimizations to deal with that level of optimization and either take the hit, or make more things public while that compiler doesn't exist.
Let the header files be written for people to read first, and only if there is actually a big performance issue, and the problem is the interface do you need to revisit it (and I'm not just saying this - I will frequently and happily go back and modify interfaces to allow for less data movement etc, but most of the time it really isn't important).
I think you are probably right to disagree with me though - I think I should have said that it is more of a limitation on how object files work, rather than how machines work. Object files aren't the only way things can work.
With C++ you have the third option where the compiler makes sure that the "people will do the right thing" with the private keyword - assuming they're not doing some weird pointer math to access the private members..
Of course, you'll have to deal with ABI stability now but it's all tradeoffs for what your requirements are.
Of course you should do the right thing, but if you want to break the private of C++ it is much easier to "#define private public" before including the header file.
Right, but as soon as you have private stuff in your header file, that is leaking implementation details. Yes it is kind of true that these are compile time checked to make sure that people don't do the wrong thing, but it is still an implementation detail that is leaking.
It comes down to a cost benefit thing - is the cost of poorer readability worth it for mitigating the risk of people doing the wrong thing? My experience says no, other people's experience says yes. Probably we are working on different problems and with different teams.
If you want to have some abstract type where you don't let people know anything about the innards, but you do have an explicit interface which enumerates what you can do with it, then yes - you can only really pass around pointers to these things and people outside your abstraction can only pass references not values.
If you want people to be able to pass your abstract type by value (among other things), then either you need to let them know how big the thing is (an implementation detail) or you have to expose the copy implementation in such a way that it could be inlined (more implementation details).
Sometimes, the "pure abstraction" approach is best where you only ever deal with pointers to things, and other times the "let's pretend that people do the right thing" approach is best. I don't see this as a header file thing though.