To Intel's defense, there is a reason they resort to their approach of gradual improvements. In 1981, after a couple of successful iterations on their CPU, they did make a clean break with the past with iAPX 432 [1] - an entirely new 32-bit CPU designed to be programmed entirely in high-level languages. Hardly anyone seems to remember it, and for a reason: it failed, hard. Intel seems to have learned the lesson and didn't do this anymore.
A good point. I didn't cite it here because its implementation problems were severe and the reason it failed. Yet, I do give credit in more thorough discussions on INFOSEC history because the architecture was pretty awesome for Intel. Then, they watered that down with BiiN's i960MX. Then, even less radical (in security) Itanium. They lost hundreds of man years on the first, around a billion on second w/ Siemens, and at least $100 mil sunk into Itanium w/ HP.
Intel certainly tried to make up for their x86 monstrosity several times. They got smarter and smarter about how they did so in terms of market acceptance. A combination of engineering mistakes and foolish market choices (imho) made them pay. As you said, they learned to give customers the insecure garbage they wanted or risk extremely large losses. Links below for those interested in the specifics of their better work.
Interesting, though this shouldn't be an indictment on HLAs in general, rather the iAPX 432 in particular. Same way the Mach server was only the tip of the iceberg in the sphere of microkernels.
True: the LISP machines, Wirth's Lilith w/ M-code processor, ASOS embedded Ada system, JOP embedded Java processor, and Azul Systems' Vega processors show HLA's can work just fine. Even better than competing offerings in ways. :)
[1] https://en.wikipedia.org/wiki/Intel_iAPX_432