Building PaaS abstractions on top of Kubernetes is an order of magnitude easier than doing what you guys did with Cloud Foundry. Building something that can scale to 250k Containers is monumentally hard, but with K8s, it is taken care of for you: https://kubernetes.io/docs/admin/cluster-large/
If you are a large enough organization, it is quite feasible to setup Kubernetes, chose an ingress solution and then build templated configurations that generate K8s yaml flies and run your deployments with Jenkins. I am not saying it is easy, but you don’t need any expertise with bin packing algorithms and control loops, and really is in the sweet spot of “devops” engineers.
> Building PaaS abstractions on top of Kubernetes is an order of magnitude easier than doing what you guys did with Cloud Foundry
It's worth noting Kubernetes didn't exist when Cloud Foundry started. Neither did Docker. The reason Cloud Foundry built two generations of container orchestration technology (DEA/Warden and Diego/Garden) was because it was partly inspired by direct experiences of Borg, as Kubernetes was. Folks had seen the future and decided to introduce everyone else to it.
The point here is not whether sufficiently large organisations are able to build their own PaaSes. They absolutely can. Pivotal's customer roster is full of companies whose engineering organisations absolutely dwarf our own.
The question is: should you build your own? This is not a new question. Should I build my own OS? My own language? My own database? My own web framework? My own network protocol? My own logging system? My own ORM?
The general answer is: no, not really. It's not the most effective use of your time, even if it's something you'd be perfectly able to achieve.
I know Kubernetes wasn’t around when Cloud Foundry was started. That wasn’t my point. Some of your argument was that building Cloud Foundry was hard (and I agree!), therefore you need a vendor’s PaaS. That isn’t true.
If an engineering organization takes Kubernetes and adds their own tooling around it to turn it into a PaaS for their org, that isn’t in the same league as building their own Database or what you did with Cloud Foundry originally.
If you are a large enough organization, it is quite feasible to setup Kubernetes, chose an ingress solution and then build templated configurations that generate K8s yaml flies and run your deployments with Jenkins. I am not saying it is easy, but you don’t need any expertise with bin packing algorithms and control loops, and really is in the sweet spot of “devops” engineers.