I consult and I also run a small SaaS startup. All of my deploys thus far service an audience of ~100 users per month---fairly small. I git push my monolith to Heroku and call it a day. Because of that, I don't know anything about containers, Kubernetes, Docker, etc.
As a forward thinking engineer, what should I do to stay up with the times? Is it worth my time dockerizing my projects? Should I be using kubernetes when deploying my projects?
Probably not. Container services really shine when you have a lot of different teams that all need to launch services and not worry about what happens at the hardware level or where they run. This makes sense at a lot of large companies like Facebook and Google where they have a dedicated team running the container layer. GKE makes it a bit easier but then you're locked into Google Cloud.
If you're only running a few services it really doesn't make much sense to setup Kubernetes. I setup a small cluster a few weeks ago and it certainly was a lot more involved than I expected. I ended up deciding not to use it, since it didn't really add much (for my use case) and I felt like if anything went wrong I'd be in a world of hurt trying to debug it.
Took a lot more time to setup Kubernetes than my entire current deployment system which uses https://github.com/pressly/sup.
However, dockerizing your projects might not be a bad idea depending on what you're doing. I dockerize a lot of my projects since it makes it super easy to deploy when using the automatic Docker Hub builds. Though I did have a problem where Docker upgraded and make all of my existing containers unrunnable.
The core k8s, not that bad, once you've learned where to look - logs on pods, events, and kubelet logs from journalctl. That last one is important not to forget when debugging plugin controllers.
Heroku is a perfectly valid solution. If you want to keep some of the auto-build PaaS benefits whilst containerizing for (any of) lower cost at scale, running exactly the same app configuration in dev as in production, security, attaching services not available on Heroku, distribution to clients to run on their own cloud; then Docker does offer a nice solution - see gliderlabs/herokuish or redhat s2i for a familiar autobuild environment.
And despite the one-size fits all dogma of some of the kubernetes crowd, there are many solutions to docker deployment and orchestration at different scales, especially if you're looking to retain most of the benefits of Heroku and its PaaS workflow; From the low end there's AWS Elastic Beanstalk multi-docker, flynn.io, docker swarm mode, AWS ECS and logistic sugar on top of that like Remind101/Empire, convox etc., before you get to PaaS-ish kubernetes behemoths like OpenShift or flavours of Cloud Foundry.
While Pivotal does have a k8s offering called PKS, it doesn't look like it's a full on PaaS. Instead it's just some BOSH magic that stands up all the VMs required to run k8s. PCF is their PaaS offering and doesn't as yet have a tight integration with k8s as a runtime.
PKS is part of PCF, PAS (Application Service) is the new name for the PaaS. The open source foundation has done the same inclusion (CFCR is the container runtime, aka Kubo, the CFAR is the app runtime).
You can think of the Cloud Foundry foundation as focused on marketing and standardizing the application of these open source cloud technologies to business outcomes, whereas the CNCF is more of a tech advocacy foundation that is incubating a wider range of projects, sort of like an aspiring competitor to Apache. They’re complementary and it’s good we are all cooperating.
There are a few initiatives within the CF community to replace Diego with k8s, but it’s not clear there is massive benefit in doing so - definitely some for operators who get more fine grained visibility and control, but the whole point of the PaaS is to enforce constraints and opinions, whereas K8s is much less opinionated. That said the transition will likely happen over the next while. Diego took a couple of years, maybe this will be quicker.
Disclaimer, I’m a pivotal employee speaking for myself
What I've noted elsewhere is that Kubernetes never really created clear boundaries between operators and developers. Cloud Foundry and Heroku do so with very clear boundaries.
I think the open question for Project Eirini is whether a Cloud Foundry using Kubernetes as the container scheduler would expose it directly to developers. My expectation is that no, it wouldn't do so. Maybe some Kubernetes-land portals might be added (eg, Helm), but I suspect that Dr Julz has in mind to retain the `cf push` experience that's worked so well for so long.
Disclosure: I am also a Pivotal employee. I speak for myself, to myself and at myself pretty much all the time.
the distributed systems knowledge that you would pick up by learning can be valuable in designing your saas in a way that will scale if your business grows.
i may be generalizing, but it comes down to balancing your software architecture in a way that is quick for you to iterate on now vs scaling out to support an expanding business. there is a comment further up about a monolithic deployment -- great for right now but if his business grows 100x it will be difficult/expensive to scale without significant changes.
starting with the seeds in place can remove a lot of pain later, so things like containers may not be a huge win to you today, you should consider what value they may bring down the road
Designing a monolith well should allow you to separate components relatively easily, and thus scale what you need. I don't think the style of design should be specific to "microservice archicteture".
If you anticipate needing more scale or control for your monolith, you will probably eventually need to learn about containers. Look at containers-as-a-service tech which includes Docker, Heroku's continer registry, ECS, and/or Kubernetes.
If you anticipate building systems not as a monolith, look at Functions-as-a-Service.
(potentially) interestingly Heroku uses containers as part of their PAAS, so actually your workloads are using containers, they're just handling the complexity of that for you :)
Personally I think knowing something about Docker/containers is a good plan, useful for testing on-laptop dev etc, however kubernetes is a fairly complex system and if you don't need it, and you're happy with your current process, I wouldn't rush to it.
I don't think using kubernetes is useful for small projects. It's currently still a hastle, even though it has evolved a lot. Not much in terms of benefits to show for it either.
I would read up on Docker though. Having your apps in docker does provide some nice benefits like better isolation, keeps you up with newer tech and you can then easily integrate with Kubernetes. It's also rather relatively easy to achieve.
I have very similar requirements and setup to you. I dockerized from the start, but mostly to ensure consistency of dependencies among different environments (test, staging, prod), the CI server, and development environments. Lets me treat the whole app system as a single binary even while I mess around with the internal structure.
Yes. Based solely on your desire to be on top of tech and given the broad adoption by every major provider, you probably want to invest at least a couple of days.
That doesn't mean there is any impetus to transition your current production, only that it is most definitely worth a cursory understanding for the future.
As a forward thinking engineer, what should I do to stay up with the times? Is it worth my time dockerizing my projects? Should I be using kubernetes when deploying my projects?