No one feels any shame because the reason is obvious. Normal users are constantly demanding more features and not more speed. The average person just buys a new phone when things are slow. Companies don't have infinite time so when faced with the option between adding a new feature and speeding up an old one they always pick the new feature because thats worth more.
Now imagine if the people working on the lower level stuff had the same attitude. Imagine if the codec implementations were slow because we need more features. Imagine if the graphics people never gave a damn about perf, just like the "programmers" you're talking about. Imagine if your database was the main perf hog in your system etc. etc. I'm sorry, but here's an unpopular opinion: those people are just not good engineers. The increments of software development should be abstractions, not features.