Suppose I'm an application developer who is only interested in infrastructure because there needs to be some to run my stuff.
In this scenario, would I actually learn Kubernetes, or would it make more sense to go to straight to a PaaS solution? Like OpenShift (which uses k8s in the backend, I believe), or Cloud Foundry, Stackato etc.
I always get the impression that k8s has a lot of good ideas, but doesn't provide everything out-of-the-box for actually deploying complex application. How true is that?
My advice is to go directly to a PaaS. I work for Pivotal R&D in and around Cloud Foundry, so that's my personal horse. But I'd rather that you used any PaaS -- Cloud Foundry, OpenShift, Rancher -- than roll your own.
Building a platform is hard. It's really really hard. Kubernetes commoditises some of the hard bits. The community around it will progressively commoditise other aspects in time.
But PaaSes already exist, already work and either already base themselves on Kubernetes or have a roadmap to doing so.
To repeat myself: building PaaSes is hard. Hard hard hard. Collectively, Pivotal, IBM, SAP and SUSE have allocated hundreds of engineers in dozens of teams to work on Cloud Foundry. We've been at it non-stop for nearly 5 years. Pivotal spends quite literally millions of dollars per year testing the everliving daylights out of every part of it [0][1]. (Shout out to Concourse here)
I fully expect Red Hat can say the same for OpenShift.
Building PaaS abstractions on top of Kubernetes is an order of magnitude easier than doing what you guys did with Cloud Foundry. Building something that can scale to 250k Containers is monumentally hard, but with K8s, it is taken care of for you: https://kubernetes.io/docs/admin/cluster-large/
If you are a large enough organization, it is quite feasible to setup Kubernetes, chose an ingress solution and then build templated configurations that generate K8s yaml flies and run your deployments with Jenkins. I am not saying it is easy, but you don’t need any expertise with bin packing algorithms and control loops, and really is in the sweet spot of “devops” engineers.
> Building PaaS abstractions on top of Kubernetes is an order of magnitude easier than doing what you guys did with Cloud Foundry
It's worth noting Kubernetes didn't exist when Cloud Foundry started. Neither did Docker. The reason Cloud Foundry built two generations of container orchestration technology (DEA/Warden and Diego/Garden) was because it was partly inspired by direct experiences of Borg, as Kubernetes was. Folks had seen the future and decided to introduce everyone else to it.
The point here is not whether sufficiently large organisations are able to build their own PaaSes. They absolutely can. Pivotal's customer roster is full of companies whose engineering organisations absolutely dwarf our own.
The question is: should you build your own? This is not a new question. Should I build my own OS? My own language? My own database? My own web framework? My own network protocol? My own logging system? My own ORM?
The general answer is: no, not really. It's not the most effective use of your time, even if it's something you'd be perfectly able to achieve.
I know Kubernetes wasn’t around when Cloud Foundry was started. That wasn’t my point. Some of your argument was that building Cloud Foundry was hard (and I agree!), therefore you need a vendor’s PaaS. That isn’t true.
If an engineering organization takes Kubernetes and adds their own tooling around it to turn it into a PaaS for their org, that isn’t in the same league as building their own Database or what you did with Cloud Foundry originally.
> Suppose I'm an application developer who is only interested in infrastructure because there needs to be some to run my stuff.
Yes, you are the perfect candidate for a PaaS solution.
In general, unless you are spending thousands of dollars in infra per month, managing your own services is a waste of money.
Full Disclosure: I work for Red Hat Consulting in the Container and PaaS Practice, an OpenShift/Kubernetes expert group.
OpenShift's value (and its open source upstream Origin) is not just that it is a PaaS. It's a PaaS that you can install and run on many different infrastructure bases: baremetal, VM, public & private cloud, and others very soon. I personally run a couple flavors on my laptop for demo and training purposes. So maybe upstream Kubernetes is not so simple. But that is why we (Red Hat) call OpenShift "Kubernetes for the Enterprise". It means we make some sensible defaults, decisions, architecture choices and make installation supportable to get you started. However, the fact that OpenShift & Kubernetes allow you to define you application architecture in reusable and portable object definitions is it's biggest benefit, IMO. And you may not need to spend thousand of dollars in infra per month to enjoy that benefit.
You should start with a managed solution and see if it makes sense to write/move your code in k8s. If it does, awesome, keep using the managed solution. Roll your own only when you see your traffic/load increasing your costs to prohibitive levels or if you want to do something very unique.
In this scenario, would I actually learn Kubernetes, or would it make more sense to go to straight to a PaaS solution? Like OpenShift (which uses k8s in the backend, I believe), or Cloud Foundry, Stackato etc.
I always get the impression that k8s has a lot of good ideas, but doesn't provide everything out-of-the-box for actually deploying complex application. How true is that?