rustc already tells LLVM to use stack probing for sufficiently large stack frames, so it can give a short error message on stderr ("thread 'whatever' has overflowed its stack") before aborting the program. But the author is looking for a backtrace here.

The problem is that printing a backtrace is the responsibility of the program, as part of the global panic hook. But running arbitrary user code (e.g., Backtrace::capture()) during a stack overflow would be extremely unsafe, since the thread could be in the middle of some operation that does not expect to be interrupted.

The soundness criteria would likely be very similar to those for async signal handlers, which are extremely platform-dependent, so it would be a challenge to document all the safety preconditions. An alternative would be to especially bless backtrace-rs for stack overflows alongside the default panic hook, but I don't think even it can guarantee complete safety on all supported OSes.

> The problem is that printing a backtrace is the responsibility of the program, as part of the global panic hook. But running arbitrary user code (e.g., Backtrace::capture()) during a stack overflow would be extremely unsafe, since the thread could be in the middle of some operation that does not expect to be interrupted.

> The soundness criteria would likely be very similar to those for async signal handlers, which are extremely platform-dependent

This is more or less exactly the same as any stack-print-on-crash handler you might be familiar with. In theory you should be able to unwind using DWARF eh_frame data from anywhere. In practice this stuff isn't always perfect and it's much easier to rely on framepointers, if your compiler emits them. But switching to an alternative signal handler stack and unwinding your main stack is mostly worth trying -- worst case, you're already crashing.

Of course, it's far from impossible, I'm just saying it wouldn't be a walk in the park. The backtrace handling would likely have to be fixed in the runtime (probably enabled with a -Zbuild-std option) rather than being user-configurable, since there's hardly any way to start documenting which standard library features are async-signal-safe.

But even then, doing it in a POSIXly-correct way is hard: already, the standard library unsoundly uses thread-local storage in the SIGSEGV handler (not async-signal-safe per POSIX) to verify that it came from hitting the thread's guard page. backtrace-rs mostly delegates the hard work of unwinding to _Unwind_Backtrace on Unix-like systems, but it unconditionally uses a global mutex (not async-signal-safe) to protect backtrace operations on all platforms, perhaps because of dbghelp.dll on Windows.

I think _Unwind_Backtrace is async-signal-safe (since libunwind claims to be async-signal-safe, and it supposedly uses _Unwind_Backtrace), so it could probably be done. But someone would have to put a fair bit of work into both the backtrace-rs API and the SIGSEGV handler in std, especially since _Unwind_Backtrace apparently needs additional incantations to work properly from a signal handler.