I mostly agree with you, but would qualify that by saying that writing a specialized orchestrator is not that hard for a lot of cases. I have done so a couple of times.
Lots of decisions will likely end up similar to K8S - e.g. for one I ended up putting SkyDNS/CoreDNS for service discovery on top of Etcd and using a Nginx handler to route ingress - when we first upgraded our (many years old) system to doing that, K8S was not yet widely used and I was unaware K8S did the same.
But if you have a specific use case and understand it well, it can be worth it.
However, even if you have a situation like that where it's genuinely cost effective for you to write your own, a big reason why K8S might still be a better choice is that you can hire people who know K8S - if you build your own you need to worry about knowledge transfer for something very critical to your operation.
I think writing orchestrators is fun, yet I still think most people shouldn't do so for production (but write a toy one as a learning experience anyway)
A lot of things are simple to write a naive implementation of, and sometime they useful to solve a small problem, even if just to prove this is the way to go, and its worth spending on a bigger, generalized and maybe more expensive to maintain in the short term solution.
I myself have written smol messaging queues, metrics collectors, mesh router, and a few more services that once it was agreed that they had a place in our systems, we migrated to NIH software, so we could enjoy the shoulders of giants (or at least buy support for it, rather than bus factoring it on myself and another colleague).
A production quality generic orchestrator is a lot of work because it needs to handle a vast amount of different scenarios (e.g. the list of volume types supported by k8s is vast). An orchestrator tailored to specific workloads and circumstances is often not for someone with a reasonable amount of devops experience. Hence about 1k lines of straightforward code for one of the production specialised orchestrators I wrote.
You don't avoid having to account for bus factoring anything by running k8s. It's dangerous to assume so. A k8s config for a similar sized fleet would've been many times the size of the custom orchestrator (heck, the k8s config for my home test k8s-cluster is bigger than that orchestrator), and you still need to train people on the architecture. In a custom orchestrator - at least the ones I've written - it's natural to encode a lot of assumptions about architectural choices in the code, and so all our devs could read the code and know the architecture if needed, without any presumption of needing to train them on tools they were unfamiliar with.
The reason I still agree most people should just run k8s is that it's easier to hire people who understand k8s than to hire people experienced to do a good job developing a custom setup in the first place. Our devs could trivially read and understand and modify the orchestrator once it was written as needed, but that's not the same as assuming they had the experience to make the architectural choices encoded in it in the first place.
K8s is a suitable straight-jacket to stop people from making really stupid choices, and so it can substitute for a lot of expensive and rare experience, and that's good. There are still plenty of places where there's room to do custom work.
EDIT: To add, I think a useful guideline to this is: Are you writing generic code that there's an existing solution to, or is your code effectively defining policy, architecture or config in a compact way? If the former, I'd agree you need a really good reason for doing so. In the latter, much less so. E.g my custom orchestrators have been small because 90% of the functionality we needed was there in other components - often the same components k8s ended up using - but where k8s needs to provide generic apis to allow applying a generic configuration, we could encode a specific architecture in the way we structured the orchestration. The availability of those components, incidentally, has gotten better thanks to the presence of the big, complex generic orchestration solutions, making it easier than ever to do custom orchestration when it works for you.
Lots of decisions will likely end up similar to K8S - e.g. for one I ended up putting SkyDNS/CoreDNS for service discovery on top of Etcd and using a Nginx handler to route ingress - when we first upgraded our (many years old) system to doing that, K8S was not yet widely used and I was unaware K8S did the same.
But if you have a specific use case and understand it well, it can be worth it.
However, even if you have a situation like that where it's genuinely cost effective for you to write your own, a big reason why K8S might still be a better choice is that you can hire people who know K8S - if you build your own you need to worry about knowledge transfer for something very critical to your operation.
I think writing orchestrators is fun, yet I still think most people shouldn't do so for production (but write a toy one as a learning experience anyway)