Hacker News new | past | comments | ask | show | jobs | submit login
Deploying Kubernetes clusters in increasingly absurd languages (leebriggs.co.uk)
149 points by jaxxstorm on May 6, 2022 | hide | past | favorite | 68 comments



Since I wrote this article, my colleagues and friends have chipped in with even more absurd examples.

How about some Pascal? https://github.com/jaxxstorm/pulumi-examples/pull/96 Or maybe Emacs Lisp is more your flavour? https://github.com/jaxxstorm/pulumi-examples/pull/95

Or the pièce de résistance: Brainfuck: https://github.com/jaxxstorm/pulumi-examples/pull/97


TIL that the language I learned to program in (Pascal) is absurd.


Pascal seems somewhat absurd for this particular purpose but that does not at all mean it is, itself, absurd.

Though I learned on BBC BASIC so my idea of what counts as absurd as a first language may, of course, be questionable :D


it occupied a similar niche as go (easy to learn, fast to compile, easy to deploy)

honestly its a shame it fell by the wayside.


How's it feel, old man :)


I'm assuming there's some sort of transpiler to brainfuck? I just cannot stomach the idea that that was written by hand somehow....


Usual strategy is to use a macro assembler and lots of macros to build up some civilized things like stacks and subroutines. But there are also several high level (ish) languages that have compilers to BF.

https://esolangs.org/wiki/Brainfuck_code_generation


https://en.wikipedia.org/wiki/Brainfuck

> Although Brainfuck programs, especially complicated ones, are difficult to write, it is quite trivial to write an interpreter for Brainfuck in a more typical language such as C due to its simplicity. There even exist Brainfuck interpreters written in the Brainfuck language itself.


The parent comment was about a transpiler targeting brainfuck, not an interpreter. Writing a brainfuck interpreter (or compiler to x86/etc) is indeed trivial... writing a compiler targeting brainfuck is another beast entirely.


You're right of course. Although I guess some do exist:

https://github.com/arthaud/c2bf https://github.com/felko/bfpy


As my contribution, how about some sugar that DSLifies your perl implementation (should produce the exact same output as your Pulumi.pl on any perl from the last N years, tested on perl5 v24 as I wrote it):

http://hermit1.scsys.co.uk/~matthewt/pulumi

(for anybody uninterested in my perl stupidity, please skip to the 'use Pulumi;' line which is the start of the actual user written code - everything before that would in a sensible world be a library but I wanted to present the example as a single file so chose violence ;)


The ELisp version is quite nice.


Why is Emacs lisp absurd ? It’s an editor that is widely popular .


Maybe “unexpected” would be a good substitute for absurd.


It's not even that. There are several k8s packages on MELPA and they are meant for real use rather than a blog joke.


And JSON support in Emacs is not some obscure feature these days, it's heavily relied on by many popular packages.


Dynamic scoping.

Not saying it’s without utility, but to the mainstream: pants on head bonkers.


Emacs has supported lexical binding for almost a decade now and all the elisp files in Emacs itself use it. Most modern packages do too.


Bash "local" variables (advanced feature not even in the POSIX shell standard) are dynamically scoped.

Head on bonkers boots embedded systems and servers, coordinates builds, ...


Moreover, the specific example looks great.


When I worked at Google, I made an App Engine Flexible runtime for the Whitespace language (only valid characters are space, tab, etc). https://en.wikipedia.org/wiki/Whitespace_(programming_langua...

App Engine Flexible was another one of these products that just needed a container that responded on certain ports, it was quite fun to get it working. But I ended up writing a tiny "compiler" that would compile a small subset of Python to whitespace since writing just whitespace itself was quite challenging, obviously.

I wanted to release it on April Fools Day but my manager was totally against it because April fools had become its own serious entity in the marketing org and he didn't want me stealing their thunder ( these days Google has dropped April Fools jokes altogether, I guess it got a little played out).


> April fools had become its own serious entity in the marketing org

This is hilarious and not at all surprising


I remember some people being really upset with My Little Pony Borg ui theme


I understand where they were coming from, having to look at a pink/rainbow bm ui while debugging a page didn't make it any easier. But it was also one of the greatest April fools jokes of all time.

RIP My Little Borgmaster: Production is Magic


I love this article, but I also hate that Pulumi has created yet another bad DSL using YAML. One day, somebody in charge of product design will also be a good engineer, and the crazy shit customers ask for will not end up a terrible feature. Until that day...


All of this IAC crap was sold to barely-tech-literate engineering managers as “declarative makes the complexity go away!” so every IAC vendor wed themselves to YAML or HCL or JSON, but of course the complexity doesn’t go away and so the vendors came up with increasingly absurd ways to DRY up this YAML (text templates a la Helm or encoding a DSL into your config language a la Terraform and CloudFormation). Eventually people realized this was madness and what’s really needed is a programming language that can resolve to these static manifests, so these vendors in their zeal to forever misunderstand the assignment came out with CDKs so we can compile source code to YAML with a whole bunch of pointless inheritance to really make a mess of things.


I'd be curious to hear what you see as a better alternative to all of this "IAC crap."

Also how many IAC vendors are there really? Hashicorp, Pulumi and AWS? What are the others?

As I recall before cloud era infrastructure on any kind scale was still code, its just that the code was maintaining thousands of lines of Kickstart of FAI configs and associated preinstall and postinstall shell scripts. I don't remember that being any more pleasant to maintain.


“vendors” was probably the wrong word, but Helm is in effectively the same camp. I gave my solution in the comments, and it’s not Kickstart or FAI.


It’s okay they’ll all be unemployed by Christmas 2023 and we’ll look back on this tech boom with “maybe we should have solved business problems instead of spending all our time rewriting from one devops tooling to the next?”


That claim is more absurd than those you mock.


This tech bubble already popped a leak (Nov 2021) and is deflating rapidly. There’s enough runway to last another year or two. Down-rounds and lay-offs are coming fast.

Companies with ample cash will be fine, of course, but all these spurious startups with egregious VC funding, overpaid Pulumi and Kubernetes and <insert all the other unnecessary tech> engineers will soon be out of work and unable to find similar employment comp.

People who get shit done and can run lean profit driven businesses will continue to survive just fine.


I was going to complain about conflating the market with the economy, but the more I wrote, the more I started to agree with you.

Companies relying on crazy market valuations will pop just like they always did. Companies with sufficient revenue OR cash should continue.

Kubernetes is fine and it's not _unnecessary_. It's absolutely crucial to many companies and having this tech available has solved many problems. It creates other problems, of course.

People like to complain about Kubernetes, but my viewpoint is that microservices is the unnecessary tech here. Sure, some stacks may benefit, but I'm willing to wager that most companies are not doing microservices, they are actually creating distributed monoliths just to be able to ship their org chart. And, in the process, adding all sorts of technologies that shouldn't be necessary.


Yep. The freedom of choice created by microservices and developer tendency to play with everything under the sun and immediately rewrite tooling into it. Now you have people writing node alongside python alongside haskell alongside rust in the same org all talking to eachother via really inefficient http calls.

It’s not just that of course. It’s the entire web toolchain and build pipelines, SPAs, react, node/npm, that entire community reinventing “SSR” and caching.

It’s the “modern data platform” movement where people stop using databases and slap everything into python and 10 layers of ETL and job queues and dags all so it loads into a cloud data platform with a separation of storage and compute at the end.

It’s all a bunch of crap is my issue and why I call it a bubble. Nobody is solving actual business problems. They’re just creating tooling and then new companies to make that tooling less insufferable.

As one tiny example, how much of say Datadog’s revenue is coming from overfunded VC companies who have never considered cost or utility. Those logs should have been sent to /dev/null in the first place is usually the answer.


I don't suspect you'd change your mind, but I disagree with this statement:

> Nobody is solving actual business problems.

I think most people are solving business problems and that you're (perhaps justifiably) jaded on the tech side of it.


It's more like - there's a bunch of people trying solve business problems while having to work around technology decisions that they don't get to make, then hiring other people that are working on solving the problems created by said choices.

The entire CNCF doesn't solve a single business problem. And I say this while working with K8s since 2016. Not once has a customer thanked me for my tech stack.

You and the parent are probably talking about different people.


> The entire CNCF doesn't solve a single business problem.

Hmm, I disagree with that assertion. Yes, a customer isn't going to thank you for making an investment in depending on open standards. But it's often a worthy investment because it lets you hold other vendors you depend on accountable. I've seen this firsthand with OpenTelemetry and the Observability space, where it's not uncommon for vendors to lock you in and get weird and shitty with you. I'd love to live in a world where that didn't happen, but in this case I think there's a very real business problem being solved.


Eh, I don’t really have much of a problem with that. The HTTP calls are inefficient, but you don’t dispatch them in a tight loop so it’s a negligible concern. The more valid concerns are tracing calls through services, but I’ve never had much of a problem with this for whatever reason. And the alternative has its own issues, so (shrug).


We've been hearing the same exact thing since at least 2013. I remember when 2015 was going to be the next huge tech crash, but nothing happened.


https://en.wikipedia.org/wiki/Dot-com_bubble

> Low interest rates in 1998–99 facilitated an increase in start-up companies

> In 2000, the dot-com bubble burst, and many dot-com startups went out of business after burning through their venture capital and failing to become profitable.

Sound familiar? It should, because history rhymes.


How exactly does a Wikipedia entry for the 1999 dot com crash support your assertion that that a market crash is imminent? How is that any different than make posting a Wikipedia link for World War II as supporting evidence that World War 3 is imminent? At any rate it does not sound familiar. Nor does it look familiar. The world in 2022 looks very different compared to the world at the end of the last century.

People have seemingly predicted 20 of the last 1 tech bubbles. The "Dot Com Bust 2.0" headline is practically a meme at this point. I feel like it's something lazy journalists do on slow news days. Here's a brief sampling from the last 8 years.

From 2014 - "In Some Ways, It’s Looking Like 1999 in the Stock Market"[1]

From 2016 - "This Tech Bubble Is Bursting"[2]

From 2018 - "Silicon Valley tech bubble is larger than it was in 2000, and the end is coming" [3]

From 2020 - "6 Reasons This Tech Bubble Is Bursting"[4]

Ad infinitum.

[1] https://www.nytimes.com/2014/03/30/business/in-some-ways-its...

[2] https://www.wsj.com/articles/this-tech-bubble-is-bursting-14...

[3] https://www.cnbc.com/2018/05/22/tech-bubble-is-larger-than-i...

[4] https://www.fool.com/investing/2020/09/10/6-reasons-this-tec...


What is utterly absurd is not the claim that capitalism has boom or bust, but that IAC is in any way related.


stop listening to cnbc, start looking at the numbers. shit's gone really bad really fast and it's only starting. first signs of weakness in LQD, HYG and JNK have been there since december and it's been straight down from there. stay aware.


Market something irrational something solvent


Yup, the clock is ticking and a lot of people haven't realized yet. The circular economy of VC-funded startups selling things to other startups will be some of the worst hit...

Not sure about Pulumi, looks like they only have a series B 1.5 years ago. Either they have a very short runway at this point or found some real ARR.


Would you rather manually manage things through a UI? It’s not like IAC doesn’t solve problems. Sure it introduces some and isn’t optimal but I’m curious to hear your solution.


I gave you my solution. Write typescript or Go or whatever that emits the static configs that describe the desired state of the world and then feed those into the IAC engine which will diff them against the actual state of the world and make the requisite changes. It’s still “iac”, it’s just simpler.


So.... pulumi? That sounds a whole lot like how Terraform, Pulumi and kubernetes work. But you just want to roll it yourself?


No. IAC absolutely absolutely solves problems (repeatability and a source of truth you can put in source control, among others) that GUI management has no good solution for.

What OP is getting at is that IAC is often not /sold/ this way, it's sold as way to make managing complex systems simpler, in the sense that someone with less expertise can use it effectively, because marketers and salespeople love promising something that will remove all that pesky setup work and let you start on whatever it is you actually want to do. This can be true for some tools (e.g. I don't have a great deal of knowledge about compiler internals or assembly, but my compiled code still does what I wrote it to do), but not necessarily all.

Managing an AWS environment is an inherently complex task, and IAC tools can't reduce that complexity without being prescriptive. We may in the distant future get to the point where prescriptive configuration is fine (I trust that my compiler will make better assembly optimization decisions than I ever will), but we're not there now. What we have now addresses UX issues with using GUIs for these tasks.


You guys will just love the shared mentality on /r/DevOps. Apparently YAML is the one true declarative language that all others are failing to imitate. I'm literally Judas for preferring to use typescript with pulumi.


i hate yaml more than xml and that is saying something. i get looked at strange when i say this loudly, like i'm from mars or whatever. it's like being in a collective hallucination where you know you're right and everybody else is wrong and nothing makes sense anymore.

i've tried to fight this by leading by example with https://dhall-lang.org/ but apparently having a configuration compiler is too much... unless the compiler takes yaml as input, then it's fine.


This article gets one thing right: if you're going to autogenerate something as complex as YAML, you'll really, really want to stick to a sane subset of the underlying language (here, JSON).


It's a bit of an XKCD where they keep re-inventing YAML-based DSLs. We have weird "native" YAMLs for AWS, GCP and Azure, and then YAML DSLs to generate those YAML DSLs. Even SaltStack and Ansible do it...

It's almost like someone had a bad idea and put XKCD and Xzibit in a blender...


Ansible even states that it doesn't have DSL while it actually is when you go to the rabbit hole deep enough.


… you don't have to but peer slightly into that rabbit hole; your first example Ansible playbook should very clearly demonstrate it has a DSL…


It's a trade-off. In exchange for a hair-tear-outable user experience, you get the ability to (relatively) easily load and transform that data, which gives you lots of automation and integration options.

It's not unlike LISP's predilection for sexpr-based DSLs, I suppose.


why-not-both.jpg?

I hate yaml as much as the next developer, but I've met plenty of folks who far prefer it to using a "real" language. Different strokes for different folks?

The challenge from a product standpoint is how to balance these different ways to do something without (a) ostracizing one group, and (b) creating a confusing mess on the "get started" path.


Some people prefer some things just because they haven't learned a better way, but if you show them a better way, they'd gladly use it. I think the mission of a product designer should be to see through what the user thinks they need, and provide something that solves their real problem in the best way.

In the case of IaC, what the customer really wants is a programmatic way to generate instructions for Pulumi to use to build infrastructure. The dream of "declarative" anything is to explain what you want and have a machine make it reality, right? But in practice that's hard to do. The machine needs to be very, very smart.

But it's not that smart. So we cheat and invent non-programming-languages, so that people end up telling the machine in far too much detail what they want. Do I really want to tell the machine to "go in a loop creating resources as long as there are resources in a list" ? Or do I really want to do some other thing, and making these resources should just be an incidental part of that that I shouldn't have to explain at all?

Programmers who like YAML for anything other than pure data serialization don't actually like YAML, they just like that they can cheat with it. No need to write a configuration format if you read your configuration from it. No need to write a DSL if you execute logic based on it. The programmers who don't like YAML wanted to do the same things, but were bitten by all the problems that came from using the wrong thing for the wrong purpose.

So really, the product person and experienced engineer need to work together to prevent this whole situation from ever occurring. Create a schema, and create libraries that can generate YAML based on the schema, but do it in such a way that no human would ever want to edit it by hand, and too complicated to ever write a generator for it without the schema. This way the customer can't shoot themselves in the foot, or demand changes which would make everything worse.


You're speaking my language here, FWIW. I just think that the market for IaC is big enough for a product to have its cake and eat it too. Doing that well is extremely challenging, though.


https://github.com/cruise-automation/isopod

Starlark -> protos -> k8s api. Jic you wanted more confuse


I know this piece is mostly for the humor of increasingly arcane config generation systems, but given that Pulumi is attempting to displace the most popular application using HCL2, I find it interesting that the HCL example looks ergonomically awful.

The CUE support in particular looks fairly interesting, because I think it is going to end up being the universal translator for a bunch of these golang tools due to its ability to pull in types directly from code.


Might as well do MUMPS and COBOL for the next article.

I found the Fortran example very interesting. I guess it makes sense modern libraries would be available.


I actually tried to get something working in COBOL, but my lack of familiarity with it made it a struggle. Would happily take a contribution at https://github.com/jaxxstorm/pulumi-examples


Brainfuck?

https://en.wikipedia.org/wiki/Brainfuck

Example: ++++++++[>++++[>++>+++>+++>+<<<<-]>+>+>->>+[<]<-]>>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++.



HAHAHHAHAH I LOVE IT


There have been some good questions in a few of the threads here about why Pulumi added YAML support in the first place. I thought I’d share a few thoughts on that.

Certainly it wasn’t only about being able to compile all sorts of esoteric languages down into a YAML-based data representation - though it’s fun to see the results of that here :-).

Our goal at Pulumi has really been to offer the best tools for infrastructure as code, and for defining and managing cloud infrastructure more generally, for any developer working with the cloud.

We started with a focus on the high-end - teams managing significant complexity of cloud infrastructure to get the most they can out of the managed services their cloud providers are making available as building blocks. We’ve grown with this part of the market, with great adoption and usage across many of the most advanced cloud engineering teams.

In order to grow with these users, we’ve invested in many layers of the Infrastructure as Code stack. Some of those improvements have been related to the software engineering benefits of using traditional general purpose programming languages to manage cloud infrastructure - IDEs, types, abstraction and reuse, test frameworks, packaging and versioning and so much more. But we’ve also been making improvements at many other layers of the stack. Our cloud deployment orchestration engine is now, I believe, the richest option in the market - with multi-cloud support, built-in secrets management, refactoring support with aliases, rich controls over replacement behaviour, built-in support for components, and much more. And our native providers for Azure, Kubernetes, Google Cloud and AWS are the most complete and most up-to-date providers for managing those cloud platforms.

We wanted to be able to offer all these benefits to the broadest possible range of developers working in the cloud. Since we already have very rich programming language options, we wanted to add an option at the other end of the spectrum - the simplest possible interface we could to the Pulumi platform. And that is what Pulumi YAML is. It is very simple, and designed for small scale use cases (a few to a dozen resources). It composes with the rest of the Pulumi ecosystem, so it’s easy to push complexity into components built in other Pulumi languages, to reference outputs of stacks deployed by other Pulumi languages, or even to "eject" into another Pulumi language if the complexity gets too high in YAML. So unlike many other IaC ecosystems where YAML or a DSL is the only option, in Pulumi, YAML can be a nice solution for simple use cases, without having to be abused for the complex use cases that are already well served by Pulumi’s existing alternative programming language choices.

There’s more details on all of these points at https://www.pulumi.com/blog/pulumi-yaml/ for those interested in learning more!


What is that cat replacement you are using?


It's called bat ( https://github.com/sharkdp/bat )




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: