I think the kubernetes project is heavily driven by Google marketing, and that they are not doing this out of charity, but because they are trying to get you to use their cloud platform in the long run. They know getting somebody to build their stuff on open-source kubernetes is a win for them. After some time people will realize that running kubernetes yourself is actually harder and more fragile than just running your app without it and at that point the obvious move will be to use a hosted kubernetes service, like Google's cloud.
And I really think just handing over all our apps to google to run them for us is not in our (developers) interest in the long term. It would further solidify Google's "monopoly of the internet" position and also means that in the future - once we have succeeded in convincing our bosses to just rent every interesting bit of technology from google - that the only interesting jobs left will be... at Google.
So please go ahead and downvote me, but please also try to consider my point of view (that there is a ton of very aggressive marketing with a financial incentive going on here) next time you read and defend some kubernetes hype piece.
You are correct on Google self-interest in the matter. Not so sure that Google's self-interest is so divergent from the average developer interest.
Kubernetes is hard to run by oneself. Worse, self-cobbled setups are also hard to run by oneself. Distributed systems, that is systems that require uptime and/or more than one computer, are hard.
The fact of the matter is that computing is the new electricity. While one can run oneself's own generator in the backyard, it is more efficient to buy electricity from a big provider, small-scale solar notwithstanding.
Kubernetes offers a well designed abstraction that simplifies managing distributed systems at most scales. Bonus, it has become a shared standard among providers of computing. That is a big step beyond proprietary lock-in abstractions [AWS, etc.], and likely close to the best one can hope for, technically speaking. Economically, perhaps a future solution is regulating computing providers as utilities.
I agree that running kubernetes ourselves does actually serve our own interests. But I still think it's much more of a slippery slope than alternative solutions (being that some of them are not developed with the intention to eventually get you to use the hosted version)
Which part of kubernetes is "developed with the intention to eventually get you to use the hosted version"?
Offering software for free, software that is widely adopted cannot be based on the motivations you propose, unless there is a grand conspiracy of cloud providers for which I am unaware.
LXC on which Docker was based always had a much more sensible container model and contrary to the pervasive misinformation by the devops ecosystem is far easier to use.
It supports a standard OS environment and does not enforce the use of layers so users don't have to deal with single process non standard OS environment which removes half the complexity.
We are working on a project, Flockport, that supports LXC and provides an app store, orchestration, networking, distributed storage, service discovery and HA. So there are attempts to explore simpler alternatives.
I see a semblance to their original motivations in building chrome. By making the web as a whole more competitive, it made google's offerings more competitive (more ad surfaces, more gmail users, less OS lockin). Actually winning the browser war isn't necessary.
For a huge number or organizations, the cloud isn't a meaningful option because the switching costs are astronomical. Kubernetes, otoh, means someone that chooses on prem today actually has a migration path later.
Google benefits hugely from that path just existing.
I think Google's interests are aligned with developer interests here. The main benefit to GCP that k8s success brings is that migration between clouds becomes easier (e.g. moving from EKS to GKE would presumably be easier than moving from Elastic Beanstalk to App Engine). Less lock-in benefits everyone except the current market leader.
K8s only gives you theoretical leverage in negotiations with "cloud providers". As long as you can't reasonably run it on your own hardware (because it's simply way too complex and you'll be having trouble finding experts, even at obscene salaries) you won't be realistically able to migrate to another "cloud provider" either.
It's an old trick from the playbook of ERP software vendors: make your software absurdly configurable so that all the meat is in the configuration.
Worse (and I know this isn't popular and will hurt) k8s for all intents and purposes just runs Docker images which are just pointless wrappers that don't add anything but point-of-failures and vulnerabilities.
Don't think the term "container" is really well-defined.
The "container" that docker and others implement is actually a collection of different kernel namespacing features. I assume the one you are referring to are cgroups. I think a better description would be that each process in a linux system is part of (many) cgroup hierarchies. And you can have more than one process in each of the groups.
I think what parent meant is that you can actually get all of these really nice isolation features for your service without using "Docker". It is trivial to enable them using linux command line utils, or use something like systemd which can also do it for you.
Docker and cgroups/namespaces really only isolates you from mixed shared libraries situations, by essentially pinning your lib in the Docker image. That is, by ignoring the problem of stale/insecure shared libs, which is the entire point of using shared libs in the first place.
Docker doesn't isolate you from resource exhaustion (out of memory or files, infinite loops), from incompatibilities of the host kernel and Docker version bumps (so your shiny image isn't guaranteed to work on newer kernels and Docker versions), and makes it impossible to use host user identities and permissions. Thus projects tend to avoid plain regular file access, using databases and block devices and what not as workarounds.
IMHO Docker is an anti-pattern to "solve" an incidental problem of your own making.
My main argument was against a container being something "more" than just a standard linux process in the sense that "more" is extra software that wouldn't otherwise be ran.
Of course Docker is just the popular brand name for those isolation features and 'image' 'container' and all related terminology exist independently without Docker
Docker is also a cross platform command line tool that helps to manage said images and automate a lot of work that would otherwise need to be done. For me it is kind of like Dropbox: yes you can piece it together yourself, but using it is very convenient and you can spend your time elsewhere.
In my opinion its on its way out. CNCF and k8s has already basically replaced it with the CRI initiative and containerd[0][1]. Runc, rkt, and a multitude of other tools run containers. img, ocra-build, and others can build them.
[0] I realize this is from Docker as well but I feel it supports my point that Docker, Inc themselves are shedding baggage to still stay relevant.
I'll have to look into it thanks. I'd like to not end up requiring different tools to run the same container on Windows/Mac and Linux. Currently my workflow is easy, I create my Dockerfiles on a mac, do all of the building and testing, and then just tell devs on Windows to pull a repo and do a docker-compose --build up. I hope in the future this will not grow into tool hunting.
You'll probably be able to continue using the same tools across platforms to run containers.
I personally feel that Docker tries to do too much, almost the systemd of the container world. I believe having alternative container runtimes and build systems decoupled from Docker (both in the running program sense but also the company) will be the best in the long run.
With or without docker your workflow will remain the same. Its the image itself (the CRI spec) that makes that cross-platform magic work. I myself do my development on a Windows machine, ship a tar.gz off to Google's Cloud Builder to build the image and publish to a registry which then gets tested and debugged on a linux host.
Well I agree that the interests are aligned insofar as Google is trying do dethrone AWS which currently has a near-monopoly on cloud. But I'd still rather live in a world where the compute fabric of the internet is not centralized in the hand of a few corporations. I'd much rather get paid a salary from money that would have otherwise gone to a monthly *-Cloud payment. And I believe that in a lot of cases, the non-Cloud version of the product would actually be a better, cheaper, more reliable solution. But that will become a harder and harder sell the more successful the marketing from cloud providers is.
You seem to imply that k8s somehow means having all your hosting and deployment provided by Google, which seems pretty fallacious.
Insofar as it gets people to use GCP, it's not necessarily very useful in that regard. Other replies have pointed out that k8s provides a useful abstraction above multiple cloud hosting solution providers. I'd like to add several reasons.
1. Alternative solutions exist. For example, Docker Swarm. It seems to me that Swarm, otoh, is too inflexible for mainstream popularity (?).
2. Alternative hosting solutions exist: why GCP and not, say, AWS?
3. Some companies build on top of the main k8s features and provide their own solutions tied to their own offerings, obviating the use of GCP. For example, rkt and OpenShift.
So even if Google is the dominant interest to the extent that k8s would fail if it stopped its contributions to the project, it is not obvious that Google will be able to capture all the value that comes out of the k8s project (of course, it expects to capture enough of it to make fiscal sense)
Could you expound on that a bit? I'm kind of new to the whole container thing and have been making an effort to educate myself, but it's not clear to me what's "wrong" with Docker Swarm that Kubernetes is so much more popular, especially when it seems like everyone is using Kubernetes with Docker already.
Alright, I haven't really delved deep into Swarm and this reply is based on ~15 mins Googling but...
1. Kubernetes since last year has first-class support for role-based access control to elements in the cluster, whereas I can't find access control concepts in Swarm.
2. I have a Kubernetes cluster where the pods simply assume and communicate in HTTP. Within the network, both Kubernetes and Swarm encrypt internal network traffic with its own CA, IIRC. Nevertheless, Kubernetes has an Ingress concept, which allows me to translate external HTTP connections over TLS into the cluster into HTTP connections (and if load is rerouted, re-encrypted with the internal-use certs, of course). This enables my containers to be focused and agnostic about the whole certificate shabang.
All major cloud providers have/will have managed kubernetes and they are free to use the name, therefore I doubt it could be regarded as direct marketing for GCP. I would expect many to consider GCP but I just don't perceive this as a backdoor way of Google getting your apps, given the broad buy-in by the cloud provider community.
Well yes, you can replace Google with like two or three other companies in my comment. At the end of the day I don't like the trend of outsourcing all the interesting bits of technology to a handful of mega corporations. I think it is not a good bargaining position for us developers, Google and the like can already get away with paying shitty salaries and it will only get worse the more control they have. For all the smaller companies, they are paying a cloud provider instead of hiring local talent to build solutions. It's loose/loose.
That's effectively anti-globalization. Adapt and overcome. Good developers will always have a job in the forseeable decades to come, weak or stagnant developers will be at risk, it is a maturing industry. White knuckling tech by resisting change/momentum is no more effective than starting a trade war with tariffs.
That is a very interesting counterpoint! I guess I agree with you if the choice has to be between {Google, Amazon} or "only Amazon". But I'm advocating for the third choice, where we continue (or go back) to running our own infrastructure.
If running your own infrastructure is a strongly desired option, people will do the work to make it easier to host your own k8s cluster. I agree it's not there yet (I'm the primary person for our team running several self-hosted on-prem k8s clusters) but I can see work is already happening to make it easier.
But I don't want to run my own infrastructure. Infrastructure is very hard to do well and the problems it poses are not necessarily problems that developers want to tackle. It also requires dedicated resources and staff that many smaller orgs have difficulty maintaining.
I'm fairly sure that's the main reason why AWS and GCP are so popular, not some nefarious marketing conspiracy.
MS is also investing heavily in Kubernetes and their offerings on Azure is actually pretty nice. I've run k8s and OpenShift on premise, I've managed 'own infrastructure' - and no, I don't want to continue/go back to that. You cannot do that well and at that price on small-scale. If you think you can, you're doing something wrong.
People will host their own infrastructure if it's the better cost decision for them. No one is stopping a company or an individual from going the self hosted route.
EDIT: I guess you could advocate against Google, Amazon, Microsoft in general for anti monopoly purposes, but that's not the same as using a cloud provider. Granted, those three are the top three offerings.
Having one person only managing 50 kube nodes IS the definition of too hard to operate. Operating 50 linux servers with something like docker swarm would be a no-brainer.
don't believe you will get down-voted for this point of view.
running and most-important keeping a k8s cluster up-to-date is a major time investment. I think the important time is balancing the benefits you get from k8s with the time draining aspect of having to run the cluster.
also, as counter-argument to what you are saying: if I'm going to move to the Cloud, why leverage k8s at all? Most clouds have other abstractions that you can use when building if you don't mind being locked into a specific bigCo.
Use k8s exactly for the reason of either being federated across cloud providers or simply portable across cloud providers. You can't do that with vendor specific solutions e.g. elastic beanstalk
What keeps people highly coupled with could provider is not where their code is running but rather other "here only" solutions like custom DBs or infrastructure.
You can rewrite code deployment script for any provider in few days worth of work, but switching between other tech stack will be near impossible.
> You can rewrite code deployment script for any provider in few days worth of work
I've directly experienced the counter example to this at a major SaaS company. Beyond methods you'd use for < 20 machines this just isn't true. Your orchestration of machine lifecycles will be tied up into the particular cloud no matter what kinds of abstractions you try to put in. Unless you start running on multiple clouds or you write to k8s instead of the cloud APIs, you will be locked in.
> What keeps people highly coupled with could provider is not where their code is running but rather other "here only" solutions like custom DBs or infrastructure.
I agree, and the advent of k8s adoption is one less thing that locks you into a provider.
Of course they aren't doing it out of charity. They're doing it to prevent an AWS monopoly and to ensure that their offering is the preferred one.
And we'd have cause for concern if 1) Kubernetes becomes dominant and 2) Hasn't accumulated a critical mass of non-Google developers.
That's what happened to Android, and why Google is very successfully closing it up again.
But neither applies to Kubernetes. AWS is still dominant so Google is still incentivized to keep Kubernetes as open as possible.
But more importantly, Kubernetes is much more than a Google project now. Google could pull all of its developers off Kubernetes today and it would keep going just fine.
Every choice in life is a set of trade-offs. Each cloud provider (AWS, Oracle, IBM, Microsoft, Google) is likely to embrace some form of Kubernetes or Kubernetes-ish container orchestration. And each of them is likely to end up in a situation where there is friction if you want to switch away should build on top of their offerings.
But that's kind of to be expected. It reminds me of using .NET and ODBC/ADO and for so long there was this "benefit" that went like this, "But you can switch your underlying database anytime you want without code changes!" And yet that is rarely true at face value and furthermore, how many people are switching their underlying databases from one to another on any sort of regular basis. Who cares about the benefit of being free from lock-in if you never switch anyways!
Upvoted. Google is like a black hole, gravity pulls you in. Imagine Google disappears tomorrow, what then? Today we have for containers ECS, for email iCloud, for search Bing. Tomorrow? ...
The only part missing in most Kubernetes write ups is that you probably don't need it. I'm hopeful most engineers recognise this, although my day to day experience paints a darker picture. I know of at least one hot startup deploying Kubernetes just to attract engineering talent.
Yeah +1. I was super skeptical 1,5 months ago when I started my new job on a 22 node GKE cluster. "Surely K8s is overkill", "could've just used Nomad + Consul + Vault + X", "why not just use Fargate", "K8s seems like a distributed systemd", etc pp. Long story short, K8s on GKE just works, saves us tons of time, makes service-oriented architectures/microservices easy as pie and has led to a way better service level than the previous setup (Chef deployed on baremetal).
Yes, it probably is overkill for your 10req/s app, granted. But I highly advise everyone to at least give it a try out of curiosity, because a lot of hard problems at medium scale and above just go poof with K8s.
> because a lot of hard problems at medium scale and above just go poof with K8s.
No, they don't? I don't know why anybody would just assume that something as complex as kubernetes would just run flawlessly once you actually try to run it on thousands of servers. Must be something to do with google PR because people definitely don't seem to assume the same for e.g. hadoop or openstack. Make a guess at how many people large companies have to employ to actually keep their smart cluster scheduler running?
>when I started my new job on a 22 node GKE cluster
>problems at medium scale
vs
>Make a guess at how many people large companies have to employ to actually keep their smart cluster scheduler running
K8s obviously is not a silver bullet and of course there's ops work to be done. I can't comment on whether operating K8s clusters in other contexts makes economic sense, but I know it does for our current team.
> makes economic sense [...] for our current team.
But the reason for that is not that it makes "hard problems go poof at scale". The reason is that you're using a hosted service where somebody else (in this case, Google themselves) takes care of the problems for you for a fee and - at small scale - you only have to pay them a fraction of a single operation's engineers salary for it.
So of course it makes economic sense for you to use a hosted service where the sharing economy kicks in, but your recommendation to use kubernetes because it solves hard technical problems at medium scale does not follow from that.
> [Google themselves] takes care of the problems for you for a fee...
GKE's services are, as far as I can tell from their pricing page[0], free. The compute running on top of GKE is charged at the GCE rate, the master node spun up by GKE is free.
Disclaimer: I work at Google Cloud, but nowhere near these offerings.
How does this relate to my point that you're transitively paying somebody else to do ops for you? Maybe google's pricing model rolls this into the normal VM price? Or maybe it's currently offered at a loss to gain traction? Or are you saying they are not paying the SREs anymore and they work for free now?
Per their pricing, GCP doesn't charge for the master node VM that does the orchestration - you just get charged the normal price for VMs as if you'd provisioned them yourself. Thus, given the price of GKE is $0, the only thing I could see you "transitively paying somebody else" is experience - Google engineers become more versed in running managed Kubernetes and you don't. If the fee you perceive is dependency I'd agree - but I'd also opine that many startups/SMBs would be willing to accept that fee rather than onboard engineers to solve it.
As to why they're taking the loss on the master node VM, I don't know. I had previously expected that it was a cost and was quite frankly pleasantly surprised that it wasn't - it seems like the most obvious sell. If I had to guess as to why it's not my best assumption would be that there's far more to be gained in getting companies comfortable with scaling and from angles that go beyond just the strict monetary benefit of them going from 3 compute instances to 300.
If you are running on Google computer anyway, then I don't think you pay anything extra for gke. As far as I can tell, you are only paying for the VMs that you use. So the price for hosted k8s is already baked in whether you use it or not.
Yes, when you use GKE Google is operating K8s for you. Now ask yourself why Google and AWS and Azure offer K8s but they don't offer Swarm, Mesos, Nomad, etc. Maybe K8s solves some problems after all.
No it doesn't. You can not extrapolate from the fact that the hosted version "just works" that kubernetes at scale would also "just work" (kubernetes being the open source product that you run yourself here). Especially if the hosted version is offered by google and everybody knows that they employ truckloads of very good engineers keeping their stuff running.
The original comment asked “what about hosted kubernetes?” and the reply was discussing the benefits of that hosted solution. They didn’t say that running your own cluster would be the same. They didn’t even imply it.
Even if they had, it doesn’t justify your aggression. Maybe take a break from the internet for a bit to clear your head.
Well I am sorry if you felt the wording of my comments was too "aggressive"; personal angle is weird though. Also I think you are trying pretty hard to misunderstand what I am trying to say. Obviously, using google's hosted version will make all your problems go "poof" at any scale, because they are Google's problems now. But that is precisely the point I am trying to make: It's not a feature of using kubernetes - it's a feature of paying Google!
Or Azure. Or AWS. Or probably in the future pretty much anyone. If you want to operate your own cluster you will be able to, for everyone else it will come from their cloud provider as a "cost of getting your business." This is a great thing.
Yes, and that was their point: That the hosted solution did that. They didn’t say it was a feature of kubernetes. The entire thread started with a question about the hosted solutions and they talked about their experiences with that. Can you specifically point out where they say the hosted version’s benefit should translate to the self-hosted version?
Would it be accurate to describe some of the K8S ecosystem as building out equivalent services (monitoring, failover, health checks) to the cloud providers on top of a infra-agnostic layer?
I'm another happy user of this approach. While I generally agree with other posters all over these comments saying it isn't worth it for small or even medium deployments, I'm happy using it with a team of 2-3 for a single client.
Much different use-case than most, though: automated machine learning pipelines via Pachyderm [1]. Would never have touched K8's if it wasn't for Pachd, and between their layer and GKE it's been quite painless. May have given up on Pachyderm if it wasn't for GKE; we're not cut out for maintaining K8's, and our first shot on AWS (pre-AKS) was not pretty.
tl;dr: K8's are great when it's all but completely abstracted away from you
I see a lot of value in Kubernetes. For one thing I don't want to wake up at 1 AM if one of my node goes down and brings down with it important containers. If I scaled the environment with excess capacity then Kubernetes should start up new pods on the remaining nodes.
That is an awesome feature of K8s and similar orchestators. It is however just an HA (high availabilty) problem already solved many times over in many ways. Almost every PaaS system has it. All cloud providers have it. Getting K8s (or any of its competitors) for HA is total overkill.
We use a GKE cluster with dozens of preemptible/spot instances. Great savings for no extra management fee, and with a nice standard kubectl interface for deployments and changes.
VMs are the most reliable thing of any cloud, and often more reliable then their own managed services because of complexity. GKE does live-migration of VMs and Azure/AWS are close to the same thing, so building on that foundation and doing your own "managed" layer via Kubernetes provides a better experience, while also giving you that spot instance discount that you wouldn't get otherwise with PaaS.
Unless you're renting from multiple $10/month services, your service can be only as reliable as your service provider, which is probably less than AWS.
You hit it on the head. Weave Net is participating in the Istio project. I hope they can bring WeaveNet to Istio. That might be able to provide a multi cloud vendor service mesh.
But that’s easily possible – you can use all the French hosters at once. Latency between them is practically nil because their data centers are only a few hundred meters away from each other.
Sure, in case all of Paris goes down, you’re down, too. But in day-to-day usage you can have good reliability on a shoestring budget.
Nobody wants to wake up at 1AM because their singly-homed service just went down. Kubernetes might be a fine tool to achieve that, but I want to point out that there are much simpler failover solutions available. Failover is something you should definitely have, but also something you definitely don't need kubernetes for.
This feature won't even work for anything designed for fault tolerance. I believe it can only work for things that either rely on other properly fault tolerant services or that are completely stateless, idempotent, so they can be retried and eventually consistent (in which case the state still has to be handled by properly fault tolerant systems). Either way kubernetes cannot help here.
You mean a failover solution? It's hard to give a list here because it is such a large space and it completely depends on your product/application. It is more like a category of use cases than one specific use case.
Some ideas for things you could do in a web/website/internet context assuming you have a single point of presence:
One type of "HA" is network-level failover; haproxy (L7), nginx (L7) and pacemaker (usually L3) seem to be very popular options, but I think there are dozens of other alternatives. In terms of network-level failover, things get more interesting once you are running in multiple locations, have more than one physical uplink to the internet or do the routing for your IP block yourself.
For application-level failover and HA, one option for client/server style applications is to move all state into a database which has some form of HA/failover built in (so pretty much every DB). I think this is very common for web applications and also for some intranet applications.
Assuming you have a more complex application running in a datacenter, there is also a lot of very interesting stuff you can do using a lock service like Zookeeper or etcd or really almost any database. Of course, you can also make your app highly available without using an external service; there is a mountain of failover/HA research and algorithms that you can put directly into your app (2pc, paxos, raft, etc). Of course all of these require some cooperation/knowledge from the application. For some apps it might be very hard to make them "HA" without relying on an external system, but for some apps it will be trivial.
Note that when you move away from a web/datacenter context, to something like telecommunication or industrial automation, so something that doesn't run as a process in a datacenter but is implemented {as, on} hardware in the field, failover and high availability will have an entirely different meaning and will be done in a totally different way.
Most single-instance, single-zone failover scenarios can be handled with shell scripts, the AWS API, and cron.
But the parent's comment is missing the point. K8s is not for failover. K8s is literally just a giant monolith of microservices for running microservices. It's not intended to provide failover for non-microservices, it's intended only to run microservices, and as a side-effect of needing to be able to scale them, it inherently provides some failover mechanisms.
FROM golang:1.8-alpine
ADD . /go/src/hello-app
RUN go install hello-app
FROM alpine:latest
COPY --from=0 /go/bin/hello-app .
ENV PORT 8080
CMD ["./hello-app"]
The difference in simplicity is not in the interface that is presented to you as a user. The difference is that your shell script will have a couple hundred lines of code, while the docker and kubectl commands from above will pull in literally hundreds of thousands of lines of additional code (and therefore complexity) into your system.
I'm not saying that is a bad thing by itself, but there definitely is huge amount of added complexity behind the scenes.
That's nice and all as long as it works. If there are any problems with it (network, auth, whatever) have fun with even diagnosing the bottomless complexity behind your innocous lil' kubectl command.
The error messages are quite good. There are extensive logs. There are plenty of probes in the system. Just the other day we `kubectl exec <pod_name> /bin/bash` to test network connectivity for a new service.
To the best of my exposure, Kubernetes is a well engineered system.
The point is that automation software for admin tasks is a zero-sum game: the more a tool does, the less your devops staff will be able to leverage their know-how. The more magic your orchestration does, the less your ops will be able to fix prod problems. And for getting anything done with k8s you'll need a whole staff of very expensive experts.
Look at it this way. Kubernetes is a Tesla Model X, and scripts/aws/cron is an electric scooter.
You can try to go cross-country with either of them. One was engineered to survive extreme environments, protect you from the elements, and move really fast. The other was engineered to leave you exposed, operate in a moderate climate, and go much slower.
If you have problems with the former, it's time to call the technicians. If you have problems with the latter, you might fix it yourself with a pocket screwdriver.
That was my point. I wanted to point out that while some people have only/first heard about failover in the context of kubernetes, it is not something that is specific to kubernetes or even the problem that kubernetes was build to solve.
Of course it is not designed to be a failover solution specifically and using it (exclusively) as such would be ill-advised; I was just trying to be diplomatic while pointing that out.
We're using elastic beanstalk currently (multi docker conatiner). Not that I'm advocating it and I'm really interested in k8s now that aws has eks, but ebs is really simple to use for a simple setup.
It’s probably referring to people using Chef cookbooks blindly without any knowledge of how to configure or maintain the service itself.
I have interacted with people who run production workloads via Helm but struggle to do very basic things with kubectl (like recreating a service).
This is an argument that can be used for any high level abstraction, however. I guess Chef is used since they have very good testing tools so Cookbooks are relatively more stable than other CM definitions. Until things go wrong with the service that is.
That community sourced cookbooks are security nightmares or under maintained. I cannot see how helm charts can stay quality without a curated app store.
Thanks for the clarification. The official Helm charts go through a thorough community review process. We (Bitnami) are committed to making high-quality, up-to-date Helm charts. You can check https://hub.kubeapps.com
You probably don't need the cloud either, but everyone recognizes that it is a productivity gain. To me, this is the analogous case for K8s, you can probably do the same with VMs, but everything is a lot more productive with containers and K8s.
I worked at a place that was deploying K8s as I was leaving and my experience was that K8s was complicated and no one knew exactly how to use it or why we should use it, other than it seemed to be what everyone else was doing. I didn't notice any operational improvements over our prior system (which was largely ad hoc), but the kinks may have been worked out by now.
If you read the Innovators Dillemma, there is a very clear chapter about how new technologies don't really deliver their promises immediately; but as they're adopted and operators/users learn how to work with it, that's when productivity changes are very much visible. It took quite some time for AWS to be mainstream and then to be adopted and operated in a way that delivered high productivity gains, and it seems to be the same for k8s right now.
I really wonder about the causality of that, is it changing things to adapt the new technology makes you clean up / refresh / fix known issues along the way or is it the new technology itself causes less issues down the track
That doesn't really make sense. If you're moving to a new technology, even if you clean up/refresh/ fix known issues along the way, you are likely to accumulate similar such issues in the new technology as well, which would lead to a decrease in productivity. But that has never really been observed.
While I agree that the cleanup can inevitably lead to productivity gains, I believe they are overshadowed by the productivity gains from the actual change that is being made.
That only makes sense when seen as an obstacle to initial adoption from an innovators and sales point of view and could be said just about every innovation. But obviously not every IT innovation makes it.
It depends on how frequently you need to spin up new containers.
As someone on Azure, I think the solution for small-ish guys like me (who still need like 8 VMs in production to run our site due to special requirements) is a greatly simplified container management system. Something that fulfills the basic Kubernetes features, but isn't such a steep learning curve.
I think Microsoft is doing this with Azure Container Instances, but I haven't tried it yet. (No doubt AWS is doing something about it as well.)
Or they're going to take care of all the k8 configuration crap through some neat Visual Studio Tools. In a VS preview I saw a "deploy to Kubernetes" button. I just want something that will give me more web API instances when my server is getting slammed.
AWS ElasticBeanstalk or Google App Engine flex or something from Heroku is what you are talking about. Container management but simplified for the common use case. You can implement the same thing with k8s, but why bother for small stuff. The real value in k8s is large, company-wide installations where everyone shares each others hardware. Even then, I've heard anecdotes that kubernetes doesn't scale to the point Borg or whatever other proprietary system does. Then again, those are niche use cases to begin with.
Unless they've reduced the price, Azure Container Instances is mainly for one-off/bursty tasks. If you need a service running 24/7, it's prohibitively expensive.
I think once their managed kubernetes (AKS) hits GA, the learning curve will be slightly lower.
> You probably don't need the cloud either, but everyone recognizes that it is a productivity gain.
Care to quantify that? Because we don't use the cloud (in fact, in my current role, we don't even make use of virtualisation for production infrastructure) and we have no issues with productivity. Or rather, let's be a little more honest, we have no issues with productivity that would be solved by a move to AWS/Azure/$_some_other_cloud_provider.
Just because you "have no issues with productivity", doesn't necessarily mean that you couldn't be more productive. In my experience it takes a move to the cloud to actually see where those gains lie.
Disclaimer: I have no idea about your actual real-life situation. This is vague and hand wavy on purpose.
Indeed it is not meant to be any kind of business justification - I clearly cannot comment your business and its needs :)
In my experience of moving applications to the cloud, there are always folks who don't believe there is any gain to be made. These folks have always been wrong (so far!)
I suppose I'm only commenting because I hate to see someone snub the potential of cloud. But hey I'm just some guy on HN.
The thing is, in previous positions we've used the cloud plenty, and it's been fine. But for what I'm doing now? No way. Just not the right fit. I therefore become extremely sceptical when people cite $unspecified_but_awesome_gains from switching.
I don't have much practical experience with Kubernetes but I think one argument is that there are other alternatives to it that might give equal or better productivity gains for a lot of people.
For example Azure App Service or equivalent from the other vendors. You don't have to manage a VM.
App service is alright but I'd recommend Heroku over app service any day. However, there is some visible work being done to improve app service like supporting docker compose files to run multiple containers but the basics are still very painful.
Most notably, swapping servers (slots) always caused downtime, accessing logs was flaky and slow, azure interface is just plain slow, continuous deploy was flaky (or it just didn't kick in until 20 min later?) and I had to manually restart servers many times
i am a mainly js engineer and so far have lived entirely within the firebase world as far as my backend goes. in your (and everyone else's) opinion, then: when -do- you need it? set some quantifiable bars for me please?
I could throw the "you'll know it when you see it" but I'll try not to!
1. You have a large amount of underused hardware/VM 's you want to cluster.
2. You have an eloborate microservices system with a dynamic service discovery mechanism.
3. You have the above over many regions / teams.
4. All your services are mostly stateless.
5. Your team of rockstar ops engineers are building their own crappy version of Kubernetes just to deal with the engineering challenges your business faces.
Funnily, my current employer is just that. We predate K8s, so it can be forgiven somewhat. Transitioning, though, is just awful. We have every reason to transition, yet getting the system setup, running, and working with the current infrastructure is a daunting task.
You don't need K8s, until you do, at which point switching can be a nightmare scenario.
If you are able to be cost effective and run services on firebase, then keep doing so. Please!
If you need to do something that doesn’t map well to firebase, then my next stop would be AppEngine.
If what you want to run won’t map well to AppEngine, then start looking at K8S.
For me, this is usually wanting to run some kind of legacy system, or where a higher level of security is needed, or if you want or need to plan to run across multiple cloud vendors or in a mixed workload environment.
If AppEngine has been covered by google’s BAA when I started my current gig, I would have just used that, with k8s to handle the couple of oddball things that we are using that wouldn’t run as well there.
Personally, it’s not about scale so much as certain kinds of architectures or problems. You might have a system that needs to run a smallish number of nodes, but has reasons for needing what kubernetes offers without wanting to reinvent the wheel yet again with the existing cast of characters.
Honestly, I have no idea about Firebase or AppEngine in China. It might entirely be FUD. But, if what you imply is true, then Kubernetes is probably a good solution for your goals.
Run your application on GKE and grow your company. When it comes time to move into China, add a cluster on a cloud provider there.
When you notice that you spend such significant money (i.e. multiple salaries) on hardware capacity planning that employing a team to operate kubernetes might be cheaper instead. Probably not going to happen unless you run on hundreds to thousands of machines.
What's your bar for using webpack (or your js builder of choice)? If you're fluent and at-ease with it, you'll probably use it even for hello-world app. Same (in spirit) for k8.
In your experience with these deployments, which K8s features aren't needed? Is it the orchestration as such that's not necessary, or K8s approach wrt to the workloads deployed?
Sorry for not being clear. There is nothing wrong with K8s features. It solves a specific problem really well. However, most people & companies I interact with don't have that problem. Or that problem is just a very marginal piece of their full set of problems. Running K8s, maintaining it and making your apps work with it is pretty big undertaking. Only go there if the upside is so great it blows any alternative away.
I guess the popular term is "cargo culting". Or maybe Heroku is not hip anymore although it probably is the best advice for many companies.
I strongly disagree. I've built a couple in house PaaS type systems in the past with various degrees of success on more classical tech like straight up EC2 using a variety of Terraform/Ansible/CloudFormation/Scripting but the dev experience has always been painful compared to Kubernetes.
Developers really love working with Kubernetes. It doesn't take a lot to get started and if you're not doing anything too crazy it is a huge productivity boon IMO.
Where things get hairy is persistence, but that's the case regardless of Kubernetes or not.
Ah yes, you're right. I am sure you have a much better understanding of the requirements each of these build outs sitting behind your keyboard over there.
Heroku is still amazing indeed. Our team recently moved all Node.js based systems from AWS to Heroku and it has given us a tremendous productivity gain and reduction in complexity. Before we made that decision we spent about 8 months experimenting with K8s, only to discover it actually made things way more complex for us and reducing productivity without any clear benefits.
In that specific case, Kubernetes is approaching YAGNI territory. But as others have stated, it’s easy to get started with something like Amazon EKS and not have to worry about a thing.
Edit: Put another way, give me APIs instead of config file hell.
Look at this question and top answer from Quora. It asks how you would build a website.The answer dives into a host of engineered technologies that would suit something you expect to need to scale massively.
Wow. The top answer on that one is terrible. I would advise the exact opposite. Istio!? Nice idea, but have you tried using it? That is some serious alpha stage software almost noone needs till very late in their tech maturity.
My team runs a grand total of 1 server, 1 daemon and up to 5 concurrent batch jobs. Sometimes we bring up 1 extra server for experimentation.
It is handy to not break the bank and use GKE autoscaling for the batch jobs. Surely we could setup some batch job manager over some raw VMs, or maybe spend some time to select and learn to use yet-another AWS service.
It is handy to go to the GKE "Workloads" page and see how many servers are currently active, using how many pods. Surely we could pull that off with raw VMs, but it would take me a few minutes extra / a brittle script to pull it off.
Do we _need_ Kubernetes? No. It makes us more productive? Yes. Am I confident that in the next assignment I'll be able to reuse my skills, regardless of what VM provider they use? 90% yes, pending EKS becoming usable.
Echoes of "you don't need C, just use Assembly, it's simpler and faster" from early '90s gaming industry.
I think the recommendation is not to use another orchestration service, but to keep the infrastructure simple and not use any orchestration or service discovery at all.
Adding service discovery and container orchestration will probably not make your product better. Instead it will add more moving parts that can fail to your system and make operations more complex. So IMO a "containerized microservice architecture" is not a "feature" that you should add to your stack just because. It is a feature you should add to your stack once the benefits outweigh the costs, which IMO only happens at huge scale.
Most people know that "Google {does, invented} containers". What not so many developers seem to realize is that a Google Borg "container" is a fundamentally different thing from a Docker container. Borg containers at Google are not really a dependency management mechanism; they solve the problem of scheduling heterogenous workloads developed by tens of thousands of engineers on the same shared compute infrastructure. This however, is a problem that most companies simply do not have, as they are just not running at the required scale. At reasonable scale, buying a bunch of servers will always be cheaper than employing a Kubernetes team.
And if you do decide to add a clever cluster scheduler to your system it will probably not improve reliability, but will actually do the opposite. Even something like borg is not a panacea; you will occasionally have weird interactions from two incompatible workloads that happen to get scheduled on the same hardware, i.e. operational problems you would not have had without the "clever" scheduler. So again, unless you need it, you shouldn't use it.
I think the problem that Docker does solve for small startups is that it gives you a a repeatable and portable environment. It makes it easy to create an image of your software that you are sure will run on the target server without having talk to the sysadmins or ops departments. But you don't need kubernetes to get that. And while I can appreciate this benefit of using docker, I still think shipping a whole linux environment for every application is not the right long-term way to do it. It does not really "solve" the problem of reliable deployments on linux; it is just a (pretty nice) workaround.
Without service discovery, you end up generating configuration all over the place just to wire together your services, letting everybody know about which IP address and port where - super tedious. Add in multi-tenancy, and you're in a bunch of pain. Now try and take that setup and deploy it on-prem for customers that can't use cloud services - you rapidly want something to abstract away the cluster and its wiring at the address allocation level.
I'm not sure I understand this. You have a product that is split into many different components, and when you deploy this product to a customer site, each component runs on different hosts, so you have a bunch of wiring up of service addresses to do for every deployment?
Could something like mDNS be a lightweight solution to that problem?
And also I am genuinely curious how kubernetes would solve that. When you install kubernetes on all of these machines, don't you have to manually do the configuration for that either? So isn't it just choosing to rather configure kubernetes instead of your own application for each deployment? If it is that much simpler to setup a kubernetes cluster than your app, maybe the solution is to put some effort into the configuration management part of your product?
I'm not sure you understand the value k8s proposes based on your comments throughout this entire thread.
Managing many nodes is the reason orchestration software exists. Your suggestion to "put some effort into configuration management" is effectively naive, homegrown orchestration.
Build or buy? That's the same argument - except k8s is free and available from many providers.
Ah, if you look under the covers, Kubernetes service discovery is actually kernel-space virtual load balancers running on each node in your cluster. The DNS record just points to the "load balancer".
If your docker containers all build off of the same OS and then you run them on a host using that OS, won't the containers share the lower filesystem layers with the host?
Services use an internal tool called Apollo to describe the service, deploy it to EC2 VMs, and auto-scale. Apollo inspired AWS CodeDeploy and AWS AutoScaling.
Services are reachable via load balancer sets similar to AWS ELB/ALB/NLB.
You reach a service by the LB-set's DNS name.
If you don't want to use AWS or AWS-specific deployment tools, you could use Puppet Pipelines to do the same on Google Cloud or Azure. Puppet Pipelines was built by people who previously built some of those internal Amazon tools and offers similar functionality but cross-cloud.
And if you want even fewer moving parts, just go PaaS or serverless.
Depends on your definition of "need", it enables me to manage a ton of stuff across multiple dcs and regions easily. It would feel like the dark ages to be without it as this point.
I am writing this for folks who are reading this thread and wondering what's of said here is true and what's not true.
* Google conspiracy theories - (Google is doing it to lock everyone in).
Not true. Kubernetes was created as a best-practices fully open source Borg, the very reason it's using Docker and Etcd is the desire of Googlers to work with OSS community and it paid back by Red Hat and later CoreOs and others joining the project.
Kubernetes is complex, yes, but so is Mesos and OpenStack, and pretty much any other production grade orchestration system I've seen, so I would disregard this argument as well.
* Google is not even using Kubernetes.
Google is a big company, they can't switch to Kubernetes overnight or even in 5 years. I don't have this info, but I'm pretty sure many teams are using GKE and the internal adoption grows.
* What if Google goes away? Everyone will get locked in!
Google is not the only major contributor to Kubernetes core these days. Red Hat is another one,
and many others, so this lock in point is not valid.
Google is not even BDFL - the project is governed by CNCF (OSS foundation similar to Linux foundation) and the project was donated by Google several years ago.
Kubernetes development process is organized in the most fascinating open process I have ever seen - SIGs (special interest groups) are fully open and anyone can participate and help develop the project. I have learned a lot about openness just looking at the organisation of the dev process.
* You don't need Kubernetes if you have one service.
Not true. You can benefit from Kubernetes even if you have just one container. It solves many problems you would have to solve otherwise - service discovery, configuration management, load balancing, secrets management, fail-over, routing, publishing new releases and packaging and many others.
You probably don't want to self-host Kubernetes, because it is complex unless you have experience and desire to do so, that point is valid. But thankfully, you can use GKE, AKS or EKS nowadays.
Disclaimer:
Our company works with Kubernetes so I am biased, but we have no affiliation with Red Hat or Google.
> Google conspiracy theories - (Google is doing it to lock everyone in).
My own conspiracy theory is slightly different: Google's aim is to scorch the earth so that AWS cannot form a locked-in position.
I don't think they immediately saw this possibility, but that they moved swiftly and energetically to support the growth of Kubernetes once they did see it.
The alternatives for Google were all strategically painful. ECS might have become successful, meaning a loss for Google. Mesos or Docker Swarm might have taken off, leading to snap acquisitions of the relevant companies by Microsoft or AWS, again a downside for Google.
As the plurality contributor to the commoditising winner, Google is able to prevent the other two cloud giants from pulling ahead on container orchestration.
Disclosure: I work for Pivotal, we have a Kubernetes-based product (PKS) we co-develop with Google and VMWare.
Those are all good points, I've been surprised reading this thread at the number of people that conflate Kubernetes with Google given the CNCF stewardship of the project.
One point though on complexity is that Docker swarm is substantially less complex that Kubernetes and for many use cases could do the job.
There's a lot of FUD in the comments here about kubernetes, so let me chime in with a success story -- my company has been using kubernetes to operate a reasonably large and complex set of services (>100 nodes/pods, >5B requests/day) and we're doing it with only 3 engineers. Our experience has been great, and if I had to do it again, I'd happily choose k8s without reservation.
I consult and I also run a small SaaS startup. All of my deploys thus far service an audience of ~100 users per month---fairly small. I git push my monolith to Heroku and call it a day. Because of that, I don't know anything about containers, Kubernetes, Docker, etc.
As a forward thinking engineer, what should I do to stay up with the times? Is it worth my time dockerizing my projects? Should I be using kubernetes when deploying my projects?
Probably not. Container services really shine when you have a lot of different teams that all need to launch services and not worry about what happens at the hardware level or where they run. This makes sense at a lot of large companies like Facebook and Google where they have a dedicated team running the container layer. GKE makes it a bit easier but then you're locked into Google Cloud.
If you're only running a few services it really doesn't make much sense to setup Kubernetes. I setup a small cluster a few weeks ago and it certainly was a lot more involved than I expected. I ended up deciding not to use it, since it didn't really add much (for my use case) and I felt like if anything went wrong I'd be in a world of hurt trying to debug it.
Took a lot more time to setup Kubernetes than my entire current deployment system which uses https://github.com/pressly/sup.
However, dockerizing your projects might not be a bad idea depending on what you're doing. I dockerize a lot of my projects since it makes it super easy to deploy when using the automatic Docker Hub builds. Though I did have a problem where Docker upgraded and make all of my existing containers unrunnable.
The core k8s, not that bad, once you've learned where to look - logs on pods, events, and kubelet logs from journalctl. That last one is important not to forget when debugging plugin controllers.
Heroku is a perfectly valid solution. If you want to keep some of the auto-build PaaS benefits whilst containerizing for (any of) lower cost at scale, running exactly the same app configuration in dev as in production, security, attaching services not available on Heroku, distribution to clients to run on their own cloud; then Docker does offer a nice solution - see gliderlabs/herokuish or redhat s2i for a familiar autobuild environment.
And despite the one-size fits all dogma of some of the kubernetes crowd, there are many solutions to docker deployment and orchestration at different scales, especially if you're looking to retain most of the benefits of Heroku and its PaaS workflow; From the low end there's AWS Elastic Beanstalk multi-docker, flynn.io, docker swarm mode, AWS ECS and logistic sugar on top of that like Remind101/Empire, convox etc., before you get to PaaS-ish kubernetes behemoths like OpenShift or flavours of Cloud Foundry.
While Pivotal does have a k8s offering called PKS, it doesn't look like it's a full on PaaS. Instead it's just some BOSH magic that stands up all the VMs required to run k8s. PCF is their PaaS offering and doesn't as yet have a tight integration with k8s as a runtime.
PKS is part of PCF, PAS (Application Service) is the new name for the PaaS. The open source foundation has done the same inclusion (CFCR is the container runtime, aka Kubo, the CFAR is the app runtime).
You can think of the Cloud Foundry foundation as focused on marketing and standardizing the application of these open source cloud technologies to business outcomes, whereas the CNCF is more of a tech advocacy foundation that is incubating a wider range of projects, sort of like an aspiring competitor to Apache. They’re complementary and it’s good we are all cooperating.
There are a few initiatives within the CF community to replace Diego with k8s, but it’s not clear there is massive benefit in doing so - definitely some for operators who get more fine grained visibility and control, but the whole point of the PaaS is to enforce constraints and opinions, whereas K8s is much less opinionated. That said the transition will likely happen over the next while. Diego took a couple of years, maybe this will be quicker.
Disclaimer, I’m a pivotal employee speaking for myself
What I've noted elsewhere is that Kubernetes never really created clear boundaries between operators and developers. Cloud Foundry and Heroku do so with very clear boundaries.
I think the open question for Project Eirini is whether a Cloud Foundry using Kubernetes as the container scheduler would expose it directly to developers. My expectation is that no, it wouldn't do so. Maybe some Kubernetes-land portals might be added (eg, Helm), but I suspect that Dr Julz has in mind to retain the `cf push` experience that's worked so well for so long.
Disclosure: I am also a Pivotal employee. I speak for myself, to myself and at myself pretty much all the time.
the distributed systems knowledge that you would pick up by learning can be valuable in designing your saas in a way that will scale if your business grows.
i may be generalizing, but it comes down to balancing your software architecture in a way that is quick for you to iterate on now vs scaling out to support an expanding business. there is a comment further up about a monolithic deployment -- great for right now but if his business grows 100x it will be difficult/expensive to scale without significant changes.
starting with the seeds in place can remove a lot of pain later, so things like containers may not be a huge win to you today, you should consider what value they may bring down the road
Designing a monolith well should allow you to separate components relatively easily, and thus scale what you need. I don't think the style of design should be specific to "microservice archicteture".
If you anticipate needing more scale or control for your monolith, you will probably eventually need to learn about containers. Look at containers-as-a-service tech which includes Docker, Heroku's continer registry, ECS, and/or Kubernetes.
If you anticipate building systems not as a monolith, look at Functions-as-a-Service.
(potentially) interestingly Heroku uses containers as part of their PAAS, so actually your workloads are using containers, they're just handling the complexity of that for you :)
Personally I think knowing something about Docker/containers is a good plan, useful for testing on-laptop dev etc, however kubernetes is a fairly complex system and if you don't need it, and you're happy with your current process, I wouldn't rush to it.
I don't think using kubernetes is useful for small projects. It's currently still a hastle, even though it has evolved a lot. Not much in terms of benefits to show for it either.
I would read up on Docker though. Having your apps in docker does provide some nice benefits like better isolation, keeps you up with newer tech and you can then easily integrate with Kubernetes. It's also rather relatively easy to achieve.
I have very similar requirements and setup to you. I dockerized from the start, but mostly to ensure consistency of dependencies among different environments (test, staging, prod), the CI server, and development environments. Lets me treat the whole app system as a single binary even while I mess around with the internal structure.
Yes. Based solely on your desire to be on top of tech and given the broad adoption by every major provider, you probably want to invest at least a couple of days.
That doesn't mean there is any impetus to transition your current production, only that it is most definitely worth a cursory understanding for the future.
>It'd be nice to see more P2P cluster building tools for individuals and groups of friends.
I've been wanting to build a k8s cluster for myself and a few friends. I didn't feel there was a lack of tooling to facilitate this, but I haven't really looked into it much. Could you share some of your specific concerns, and any tips if you've done this?
There is kubeadm [1] that is part of the project and it does make it easier than in the past to get a cluster running on hosts you control. You can create VMs (Ubuntu, Debian, RHEL, etc...) and run kubeadm on each to join nodes together on the cluster.
I'd like it to be even easier though, so your friend just downloads the P2P Kubernetes application, run it and they can see your cluster and join it (with some key that you give them) using a GUI. Similar process if they are starting the cluster, except this time they name the cluster and hand out the invite (with key) to their friend.
ah okay, that's different from what I was thinking of a cluster shared by a group of friends. I'm running my on VMs from a cheap VPS provider, was going to just split the bill with others who used it.
Your use case is very interesting though. It would be interesting to see that eventually.
IMO, Kubernetes is another indicator of the success of Go (Docker is too). Go is simple and fast like C, yet safe like Python. It really combines great performance and safety.
Is it though? I believe the success of Kubernetes is mostly due to Docker whose success is mostly due to features provided by the Linux kernel, to which Docker is a frontend...
Docker's code is seems not very nice to me (when I last read it), I especially don't like the error handling, which provides very little context to go on.
Devops has added a web of complexity that distance users from the underlying technology.
Suddenly mounting filesystems is a 'storage driver', using networking technology built into the kernel is a 'network driver' or using an Nginx reverse proxy or Haproxy load balancer is an 'ingress controller'. This new vocabulary confuses rather than informs and ends up adding more layers of complexity.
What happens when things break? Simply knowing the json/yaml layers above is not going to help without an understanding of networking and the underlying technologies.
Facebook, Google, Netflix are not 2 engineers running devops, they all have unique architectures and an army of experts to run their infrastructure. The idea that you can be 'webscale' without ops experts is complete fantasy.
Maybe off topic. But before Kubernetes AND docker, how did people manage microservices? Was it a machine stack per service? Were there tools that helped keep all the services running together in a single machine? I haven't been in software for long enough to know what this past looked like so courious if any of the HN community might know more about this especially at a personal experience level.
> Were there tools that helped keep all the services running together in a single machine?
Traditionally, services on unix system are started and supervised directly by the init process ("init system"). That has been SysV init for most of the time, but in the last decade we got two major new systems: upstart (dead now) and the infamous systemd. Using systemd can still be a very good alternative to using docker today.
> Was it a machine stack per service?
That still sounds like a good approach to me today.
If your service is not big/expensive enough to need at least one full machine, why bother cutting your app up into such small services in the first place?
For example, for web applications, a traditional setup would have been to have a couple of dedicated servers that run the database and a larger pool of webservers that serve the (stateless) application. In front of that you could have two linux boxes doing the network load balancing and failover or a commercial load balancer "box" product.
I used to run a single EC2 box for each server in the arch. Config management was ansible/salt scripts which bootstrapped a server and it as an AMI. They were then deployed by ASG. A python script glued it all together.
Nowadays we do much the same thing, but with Docker on ECS. The glue script is gone, because Terraform.
It's astounding how much of Kubernetes is simply marketing. Google doesn't even use it internally.
Service Fabric[0] is leagues ahead of any cluster orchestration framework, heck, it's a complete distributed systems platform which powers all critical Azure services (Cosmos DB, Cortana, SQL DB). It is the only framework that allows native stateful applications; your state doesn't need to be delegated to another service. It offers Reliable Collections (Java and C#) which are persistent, replicated and sharded native collections.
HN tends to follow tech hype cycles, Kubernetes is entering the “trough of disillusionment” now that people are starting to use it in anger. This reminds me of prior fights like Solaris vs Linux. Kubernetes is not that bad, its actually quite elegant in many ways. It’s just that it is tackling a hard problem in the hardest way possible: by providing a new set of bottom up abstractions to replace traditional virtual machines, config management, and network and storage virtualization in the image of the Google model of massive scale workload scheduling. That means it is a whole new world to learn but you can’t quite forget about the old world yet, and need to map between the two. Is it complicated? Yeah, mainly because it’s evolving so fast and serving many audiences. It will get easier.
Simpler alternatives? Any PaaS like Heroku or Google App Engine or Cloud Foundry.
Or a serverless function platform like AWS lambda (not sure if it’s “simpler” but it is getting there)
Disclaimer I work for Pivotal who sells cloud foundry and a distro of Kubernetes
Bundle your code in a zip file. Describe it in a yaml template with the name, runtime, zip filename, memory needed, entry point in your code, and external HTTP path. Deploy it with "aws cloudformation package" and "aws cloudformation deploy".
The platform will give you load balancing, scaling, logging, metrics, internal and external endpoints, DNS, and CDN.
The PaaS platforms you mentioned are basically the same (you provide code + a bit of config and they do the rest), except priced by the hour.
The plumbing involved with Lambda tends to be onerous, as does debugging. But for many cases I completely agree it is simpler and a glimpse of the future...
I get the sense that the naysayers outweight the supporters, simply because the headline alone is not news. Don't let them deter you, spend a few hours and decide for yourself.
And I really think just handing over all our apps to google to run them for us is not in our (developers) interest in the long term. It would further solidify Google's "monopoly of the internet" position and also means that in the future - once we have succeeded in convincing our bosses to just rent every interesting bit of technology from google - that the only interesting jobs left will be... at Google.
So please go ahead and downvote me, but please also try to consider my point of view (that there is a ton of very aggressive marketing with a financial incentive going on here) next time you read and defend some kubernetes hype piece.