"clown car that fell into a gold mine" is such a brilliant image. im not familiar with twitter in those days, but I've seen other companies that definitely match this description
founder/dev on this project here - exactly!! still trying to figure out the wording between "kubernetes development environment", "kubernetes-based application deploys", "application environments in-cluster", etc...i can tell it's not quite coming across clearly in the first go, but yeah you got the idea
you can even use ingress and special URL for multiple services in a one-off deviation, but things get really messy when you want to test state-level things too like db migrations.
besides making it more convenient to handle the chain of dependencies and duplicating all the services, we also want to make it easier to test db/state migrations and larger features without having to do an expensive e2e application deploy
I think there isn't an exact term for this yet, it's a bit of a blend between ephemeral environments and skaffold. There have been a few in-house implementations at various companies where they use Istio and combinations of VirtualService and DestinationRule resources to create multi version environment and nested environments without having to change the application code, and this seems to be something like that, but local (hence the Skaffold reference).
Your project is probably going to be useful for some (who want it locally), and an inspiration for others (who want to build something similar but for their custom platform) as it can be pretty tricky in a world were we still have people doing things manually (i.e. running kubectl apply in production).
founder/developer here - yeah, in the last couple weeks I've had in the back of my mind switching the wording to "lightest-weight application deploys on Kubernetes" or something like that.
environment is an incredibly overused word for devs. development environment might even mean IntelliJ, not a k8s cluster with your dev versions inside of it
been going back and forth a lot on how to make this more clear without introducing additional confusion
founder/developer on Kardinal here - that's a good question
in reality what you're doing with namespace-based deploys or telepresence is equally "lightweight" to what you're doing with Kardinal. it's just that (at least in our phrasing), those don't quite constitute a separate "environment" as state is shared between any developers working at the same time
Kardinal matches those approaches in terms of light-weightedness, but offers state isolation guarantees too (like isolation for your dbs, queues, caches, managed services, etc). so in comparison to "ephemeral environment" approaches that give state isolation, we do believe we're doing this in the most lightweight way possible by implementing that isolation at the layer of the network request rather than by duplicating deployed resources
and thanks for the +1 on the name haha, we called it Kardinal because the goal is to deploy only "unique elements of your ephemeral environments" across the organization, i.e. number of services deployed equals the "cardinality" of service versions
I don't think thats what they meant by namespaced deploys. I think they meant an entire copy of the environemnt contained in separate namespaces.
How do you handle requests going from service A to B and then to database C vs A2 -> B -> C2?. How does service B know to route to a different database based on the version of A thats different?
service B doesn't actually know that - it happens at the level of the service mesh (you have to have a service mesh installed, right now just supporting istio but we plan on making that configurable)
we use trace headers to keep track of where the request originally came from to route it to the right database. it's transparent to service B as long as service B is properly propagating the request context from incoming request to outgoing requests
What does your writing process look like when you set out to write an entire book? I know a lot of HN users have blogs, but the idea of writing a whole book seems daunting. Would love to hear the details of your experience and what it looked like particularly for Tripping on Utopia
I wouldn't recommend my process to anyone, to be honest! I probably wrote a total of around 200k words over a four year period, of which I ended up cutting around 110k. I find it to be very true that you don't know what you actually want to say until you start writing things that aren't what you want to say. Then it's an iterative process of critiquing, rethinking, and starting over. I'm sure some people are able to start with a clear outline and then just plow through to the end, but I'm not one of them.
In terms of research, I used to keep all my photographs of archival documents, PDFs of sources, etc in DEVONThink, but I switched over to using the standard Photos app on Macs. It has automatic OCR now so I'm able to search text that appears in photographs quite easily. I did a lot of oral history interviews to supplement the archival research. This book wouldn't have been possible up until recently because it's only in the past decade that so many historical archives have been digitizing their collections. I was able to visit the key archives in person, but with others, archivists were nice enough to send me scans of key documents. Super grateful to them.
Thanks for writing this up. I’m on my own writing journey and it’s inspiring to hear that its as chaotic to others and I’m finding it (while also being productive). So cheers to that!
Personally I would like AWS et al. to die so that cloud consultants and their associates will go away and I will be able to stop being lectured about "digital transformation" and "the cloud". It's really all very boring now after seeing the same schtick several times.
points #3 and #4; "user-friendly helm chart creation" and "values.yaml is an antipattern"...I think we're just all stuck in this horrible middle ground between "need static declarative configurations for simplicity of change management/fewest chances to mess it up" and "need dynamic, sometimes even imperative logic for flexibility, configurability, and ease of development"
several commenters have mentioned Cue/Jsonnet/friends as great alternatives, others find them limiting / prefer pulumi with a general purpose language
our solution at kurtosis is another, and tilt.dev took the same route we did...adopt starlark as a balanced middle-ground between general-purpose languages and static configs. you do get the lovely experience of writing in something pythonic, but without the "oops this k8s deployment is not runnable/reproducible in other clusters because I had non-deterministic evaluation / relied on external, non-portable devices"
Ahh this is the closest one to what we originally meant haha..(founder here). but the intent was to mitigate the impact of “kurtosis” on your system by enabling engineers to see how the whole environment worked in testing (and other parts of the development process). We also have a love for taleb’s books about excelling in uncertainty, in high “Kurtosis” environments, so going with the theme made sense for us.
But I see from the comments it is decently confusing!
could be cool to see that unbundled into its own thing though