There's a compiler optimization to automatically improve icache utilization by moving rarely executed code branches far out of line, so that they don't take space in the loaded cache lines when the straight line code executes. (This still leaves the wasted disk space and possibly RAM)
GCC docs sound like the trick would be -fprofile-use, -freorder-functions and -freorder-blocks-and-partition - after a representative profiling run.
A representative profiling run for a shipping binary is a problem of course, JITs win here. DEC had a dynamic binary reoptimization framework in the 90s called DYNAMO that could do it for AOT compiled binaries.
GCC docs sound like the trick would be -fprofile-use, -freorder-functions and -freorder-blocks-and-partition - after a representative profiling run.
A representative profiling run for a shipping binary is a problem of course, JITs win here. DEC had a dynamic binary reoptimization framework in the 90s called DYNAMO that could do it for AOT compiled binaries.