> but applications sometimes know better (the ones specifically implemented by programmers who know better)
Sure, and I don’t mind it. But right now to get basic modern performance you need that knowledge. That’s my objection. I just want more projects to be able to benefit.
> See Linux's clunky NUMA lack-of-awareness and transparent huge pages blunder for examples of how sole OS control is lacking. This is not fundamentally a Linux implementation issue: the OS does not always know best.
I’m not familiar. But, I have a question. Do you think it’s possible to create a middle ground, where applications can specify a small (ideally simple) params in order to allow the OS to make meaningfully better scheduling decisions (as opposed to relinquishing responsibility entirely)?
The biggest roadblock is, as always, backwards compatibility. The second biggest is getting people to agree to standardization and documentation, solidifying the expected base of knowledge. All in all, I don't have much hope for mainstream OSes to get these improvements. But in theory it could be done. For certain workloads, applications could even just make minor adjustments like your suggestion of scheduling policy hints for great gains. This is especially about being more sympathethic about context switches, which can embody better core/NUMA socket locality, better awareness for interactivity or batch processing, better performance/efficiency core allocation, etc.. So in general I would say we could get quite far without much application programmer or user intervention.
> but applications sometimes know better (the ones specifically implemented by programmers who know better)
Sure, and I don’t mind it. But right now to get basic modern performance you need that knowledge. That’s my objection. I just want more projects to be able to benefit.
> See Linux's clunky NUMA lack-of-awareness and transparent huge pages blunder for examples of how sole OS control is lacking. This is not fundamentally a Linux implementation issue: the OS does not always know best.
I’m not familiar. But, I have a question. Do you think it’s possible to create a middle ground, where applications can specify a small (ideally simple) params in order to allow the OS to make meaningfully better scheduling decisions (as opposed to relinquishing responsibility entirely)?