95% of you reading this and thinking oh, that's neat let me use this for the company are going to waste time and resources. Why? Because the kind of complexity the article tries to address is not reached by the vast majority of the deployments out there. Typically the 'superstructure' is larger than the thing it supports. As soon as that's the case and there is not credible path to a (near!) future where you will need that superstructure you are better off with the simplest configuration that you can get away with. It will be more robust, easier to modify and easier to troubleshoot than any of these abstraction layers.
Question one for anything that you aim at production should be: "Do I really need this?" and only if the answer is a very clear yes and you're not just trying to implement $COOLTECH because you are distracted by its shininess or because 'Google does it too' then you should go ahead and implement.
My #1 technique for improving installations is to rip out unnecessary superstructure which is obscuring why things are going wrong, and more often than not is actually part of the problem. Works every time.
The same goes by the way for modeling your development process on whatever Spotify does with your 6 people development team, and in fact for any other piece of tech that you bring on board. Each of those pieces has a cost of implementation, a cost of maintenance and a cost of cognitive overhead associated with it. The best shops out there use the least number of technologies they can get away with.
We don’t have a huge system, but already we are seeing bash scripts that sed values in Kubernetes manifests or configuration stored as a dictionary inside a python script that can generate large yaml files as part of a deploy pipeline.
I think it is an inherent error in basically all orchestration tools (Kubernetes, cloud build/formation, etc), that they don’t support scripting.
For a long time I was frustrated by the lack of scripting tools for orchestration. My ideal situation would be to write the orchestration / deployment config using the same programming language the app is written in. Eventually I found Pulumi, which supports a limited number of programming languages but is basically what I was looking for, except I would like C# to be officially supported.
How has your experience been with Pulumi? I tried it back in its alpha days to compare to terraform but I didn’t make much progress due to lack of documentation and limited ecosystem.
I really like the concept. Here are the highlights:
- Helm v2 support didn't use Tiller, instead it rendered the YAML client side and pushed that to the server. Sort of how Helm v3 will be, but it didn't work very well in practice with the charts I wanted to deploy. It felt like Helm suppor t was an afterthought and the best way to deploy was to port all your Helm chart dependencies to Pulumi.
- It was really intuitive to create functions that deployed the same thing repeatedly with slightly different settings. I used it for Office365 email server settings. I managed a couple domain names that use Office365 email so I had a single function that deployed the correct settings to the specified domain name, and I just called it twice with slightly different parameters. Way more intuitive than Terraform.
We're using Pulumi for a production system and couldn't be happier.
The ability to template Yaml in Typescript and create infrastructure is mind blowing, all the time checked by the compiler. Well it's not so much templating as using Typescripts built in JSON syntax.
Using VSCode we can refactor our infrastructure code, i.e. create functions for sub-levels of our Yaml.
So we effectively combine our Kuberentes Yaml and infrastructure. It's great. Try it.
From the readme examples it seems that the code is not provider-agnostic, i.e. you write specifically for e.g. AWS. How is it then better than using AWS' native SDK for your language?
Orchestration tools (which also include chef/puppet/etc.) inherently need to be able to notice when something is just a little out of whack, and respond by just calculating the smallest action that will restore things to the way they “should” be.
Unless your scripting language is both deterministic, essentially syntactic sugar for a dependency digraph that the orchestration system knows how to calculate reduced patches of, it’ll break the “convergence” abstraction of the orchestration system: in est, it won’t know for sure what state the system is in after the script runs, so it won’t know what needs to be done to put the system back into any other state later on.
Not that such scripting languages don’t exist. Apache BEAM, for example, is a set of SDKs for various languages that let you write code that compiles to dependency+data-flow digraphs. But it’s probably not the type of scripting you’re thinking of.
I should also mention, though, that systems like k8s do allow arbitrary programming (not necessarily “scripting”) through Operators (https://coreos.com/blog/introducing-operators.html). Rather than scripting the orchestration system itself, you write a new type of component for it to orchestrate, where that component is both created/managed by the orchestration system, and, through delegation, is responsible for implementing the method by which the orchestration creates/manages other specified types of resources.
Of course, once you support scripting, why not just do everything in a deployment script?
I think a lot of these tools are based on the assumption that declarative configuration should be sufficient. The tools eschew scripting because introducing it would call their basic assumptions into question.
Kubernets doesn't support simple scripting like passing variables and such? I know that Borg supports that via borgcfg, we basically have Borg templates for everything so you rarely have to write a new one from scratch. Not sure why it wasn't ported to the outside world.
Author here. The closest thing to borgcfg in Kubernetes would be kubecfg [0]. It is based on jsonnet, the open source equivalent to GCL. The official blessed solution right now would be kustomize [1] since it is directly integrated into kubectl. I categorize both in the article with my evaluation as solutions.
How about running scripts that compile into K configs as the common task? This has been done for a long time in the DNS world where details are held in a database, some of which are then pulled out to create a new current zone file(s).
Question one for anything that you aim at production should be: "Do I really need this?" and only if the answer is a very clear yes and you're not just trying to implement $COOLTECH because you are distracted by its shininess or because 'Google does it too' then you should go ahead and implement.
My #1 technique for improving installations is to rip out unnecessary superstructure which is obscuring why things are going wrong, and more often than not is actually part of the problem. Works every time.
The same goes by the way for modeling your development process on whatever Spotify does with your 6 people development team, and in fact for any other piece of tech that you bring on board. Each of those pieces has a cost of implementation, a cost of maintenance and a cost of cognitive overhead associated with it. The best shops out there use the least number of technologies they can get away with.