I use Docker in production with Docker Compose for containers orchestration. Each service has its own Git repo with a compose file describing the orchestration. Swarm for distributed services and managing multiple servers.

Everything is a container and could be scaled, destroyed, restarted in a very simple way.

Machine creation is automated with Terraform and Ansible for provisioning, no matter the provider.

Had you tried hyper.sh? All you need is the compose file, no more terraform and ansible.

Out of interest, how are you handling persistence in your stack? Are the cluster machines homogeneous (i.e. all the same)? Can containers be deployed to any machine or must they be deployed to where the data was left?

I run a small 4 node mesos cluster. So far it's been great, took a day or two to setup and has been running for about a year now with little to no maintenance. (I should probably update). I launch long running services through Marathon and crons with Chronos. I use haproxy for service discovery and load balancing. It's mostly used for data processing, but does serve data to a production web environment (~40k req / day).

The things I like the most about mesos is that it is light weight and removes a like of the friction in deploying new services. So I can test out a new idea without a big investment or f*ing around with the giant monolith. I found container based approaches too resource intensive for my budget.

I assume the topic is Docker orchestration to automatically start/stop/restart/kill/roll upgrade/canary/bluegreen the servers.

If you are on AWS, use auto scaling groups. It achieves more than these software, it works and it's stable. (Bonus: you don't need docker at all).

If you are on GCE, use Google Container Engine (i.e. their kubernetes). That's the only [sane] way to have kubernetes running in production.

I'm planning a blog post about all that later.

That's interesting. Would you mind saying more about your kubernetes experience? I have a side-line scepticism about Kubernetes because it feels over-engineered but otherwise a container orchestration fan.

I don't get why everybody talks about "cloud" and AWS/CGE, it seems you guys ignore the fact that many companies still maintain their own infrastructure or they lease them from other companies smaller than Amazon or Google!

Because we need it of course! Either we got GCE/AWS or we gotta recreate it on bare metal with VmWare :D

If you're a [very] small company with limited resources (and limited needs?) to the point of living on a cheap featureless hosting provider, you should forget about Kubernetes and Mesos. They're hell to setup, don't waste your time with that.

What's the scale of your operations? How many servers? How much budget for hardware? How many applications? How many devs? How many ops? Maybe open another question, with more details. More people will catch up if you're lucky.

This link might help you to decide which container orchestrator/scheduler to use:

https://twitter.com/nivertech/status/777556214767124481