As far as I can tell, yes. Docker images compiled for arm64 work fine on the Macs with M1 chips without rebuilding. And as another commenter said, you can compile a binary on a Raspberry Pi 4 and move it to a EC2 graviton instance and it just works.
it will probably be a similar situation to x86, with various vendors implementing various instructions in some processors that won't be supported by all. I guess the difference is that there may be many more variants than in x86, but performance-critical code can always use runtime dispatch mechanisms to adapt.
It's true that there are extensions to x86, but 99,99% of software out there (the one you'd commonly install on Windows or find in Linux distribution repos) doesn't use those instructions or maybe just detects the features and then uses it.
I don't recall encountering a "Intel-locked" or "AMD-locked" application in more than 20 years of using x86. Ok, maybe ICC, but that one kind of makes sense :-)
That doesn't mean compilers will emit such instructions; maybe hand written assembler will become less portable if such code is making use of extensions...but that should be obvious to the authors...and probably they should have a fallback path.
I'm reasonably sure that you can take a binary compiled with GCC on a P4 back in the day and run it on the latest Zen 3 CPU.