I'm not an insider but afaik anything heavy lifting in Google is C++ or Go. There's no way you can use Python for anything heavy at Google scale, it's just too slow and bloated.
Most stuff I've seen run on k8s is not crappy webapp in Python. If anything that is less likely to be hosted in k8s.
I'm not sure why you call k8s api shitty. What is the NixOS API for "deploy an auto-scaling application with load balancing and storage"? Does NixOS manager clusters?
You can only "auto-scale" something that is horizontally scalable and trivially depends on on the number of incoming requests. I.e., "a shitty web-app". (A well designed web-app doesn't need to be "auto-scaled" because you can serve the world from three modern servers. StackOverflow only uses nine and has done so for years.)
As an obvious example, no database can be "auto-scaled". Neither can numeric methods.
If you think StackOverflow is the epitome of scale, then your view of the world is somewhat limited.
I worked for a flash sale site in 2008 that had to handle 3 million users, all trying to connect to your site simultaneously to buy a minimal supply of inventory. After 15 minutes of peak scale, traffic will scale back down by 80-90%. I am pretty sure StackOverflow never had to deal with such a problem.
Not true. Unix itself is an API for your cluster too, like the original post implies.
Personally, as a "tech lead" I use NixOS. (Yes, I am that guy.)
The point is, k8s is a shitty API because it's built only for Google's "run a huge webapp built on shitty Python scripts" use case.
Most people don't need this, what they actually want is some way for dev to pass the buck to ops in some way that PM's can track on a Gantt chart.