I love that exceptions are the classic case for this, because it's not even true that "zero-cost exceptions" are zero _runtime_ cost on the non exceptional path. The most trivial example is they block vectorization.
Well, exceptions shouldn't be the classic case here, because - as you say - they're typically not zero cost.
Zero-cost abstraction is what you get when your abstraction gets compiled away to nothing. Like properly written value wrappers in C++, or (presumably) newtype in Rust. These things disappear from final assembly.
The definition of zero-cost abstractions, as introduced in C++, was that "it can't be slower that code than the equivalent hand-written code" (e.g. code not using the abstraction).
In that regard, exceptions are interesting as if you're on the happy path (which should be 99.999% of the time - a normal program that uses exceptions as error handling method should not encounter any exception if you do a `catch throw` on an average run of the software), they can cost less than return-value-based error handling (https://nibblestew.blogspot.com/2017/01/measuring-execution-...). If you're on a "sad path", though, they will cost more.
What is pretty sure is that since compilers learned to put the "sad" path in .cold section, the code size issue has become a 100% non-issue, the "sad" path won't bloat the hot, exception-less path ; in my experience, exceptions are in cases that matter a negative-cost abstraction.