Use iptables. I know it's a glib answer, but that's how hosting always worked until AWS came along and introduced filtering from an interface outside of the control of the OS.
I was supposed to get a EASY TO USE, READY TO GO, Mesos development environment. So now I have to ssh to 5 boxes and manually add iptables rules? Why don't I just install Mesos myself then?
Honestly this whole thing just reads to me as a marketing releases, not something I would recommend anyone to use.
If the answer is 'use' iptables, fine, but then I would expect the automation that builds my cluster to do it for me, not leave me with a $200 a month cluster just waiting to get hacked.
Digital Ocean has never, since their release, provided anything impressively technical with regard to their infrastructure. It's not surprising that this is half-assed as well.
>If the answer is 'use' iptables, fine, but then I would expect the automation that builds my cluster to do it for me, not leave me with a $200 a month cluster just waiting to get hacked.
Did you confirm that the tooling is indeed missing?
I would love to see someone quantify the value in this through a tutorial or real world use case. I am Having a little bit of trouble wrapping my head around it.
Does this have enough security baked in to run a production web app? Still trying to get through the marketing speak on on the page...
Security baked in? Infrastructure services generally require that you configure and deploy your own infrastructure, including security measures. AWS still definitely allows you to very easily deploy very insecure things, and you still need to do a lot manually to tighten things up.
Now I dont have to run a $500M company to cut server costs and save dev ops time. Apache Mesos is a system predominatly used by large companies, to more efficiently utilize servers in their data centers.
The traditional way of organizing your servers requires a certain number of servers per cluster. For example: 50 webservers for your web app in one cluster, 5 servers for your database in another cluster, and 10 servers for redis, etc. On average, you'd utilize 50% of each cluster, but have the extra capacity for spikes.
Mesos makes your servers way more efficient by treating them as raw power, allowing any server to run any kind of app/task. It chops a server up into many linux containers and can shuffle around your tasks, so that a web app, redis server, and database server could all be running in different areas of the same "physical" server. I read that one company saved ~40% of their server costs and served the same load just by switching to Mesos.
Marathon is made by Mesosphere and acts as the brains or controller for your servers. You can allocate what percentage of resources go to certain apps/tasks, and it also handles deployment, scaling, failover and restarting.
Mesos supports Docker, so converting an existing app to run this on DO with Mesosphere is simple. Some applications are built for Mesos already, like Spark (MapReduce replacement), and will be inherently faster without the Docker overhead.
>Apache Mesos is a system predominatly used by large companies, to more efficiently utilize servers in their data centers.
I love Apache Mesos as much as the next person, but this isn't true. It's predominantly used by Mesos devs. It's one of the up and coming hot technologies, but it hardly has enough users to say it's predominantly used by anyone more than enthusiasts and devs.
Last time I checked very few of them relied critically on Mesos (Twitter, some parts of Netflix, and Airbnb did IIRC). Most are just test deployments or non-critical applications. (e.g. I know Ebay and Paypal are also OpenStack users because they try all the new things).
Other than those, none of those are really huge companies. I have nothing against Mesos, but I hate overhyping something when it's still clearly in an early adopter phase. The side effect of overhyping is that the tooling isn't mature/simple enough for less ambitious people so they get a bad association with the project because they tried it too early after someone implied that it was mainstream already.
>>> Mesos is a system predominatly used by large companies, to more efficiently utilize servers in their data centers.
Mesos is used by companies of all sizes in their own datacenters and in public clouds, and in production.
URX and Sailthru are merely two examples of growth stage startups that are operating on Mesos. It's not just about improved resource utilization, it's also about fault-tolerance, high availability, ease of operations and developer workflow.
How does Mesosphere differ from CoreOS? I haven't worked with either one, but as a casual observer it seems like there's a bit of overlap between the two platforms. Is that a fair assessment, and can anyone comment on the maturity of one solution vs the other?
Mesosphere is based on Apache Mesos, Marathon and other technologies. Mesos is an abstraction of the underlying hardware - exposing APIs to allow running tasks in a cluster ~ think of just launching threads on your linux box - with Mesos it's similar, you can launch processes on your servers across your datacenter or cloud. It's production software that powers Twitter, Airbnb and other companies at scale. Mesos is more than just orchestration - it's really a fabric for writing distributed systems. Apache Spark was a sample app written for Mesos. Marathon is another application that runs on top of Mesos and orchestrates containers. It can handle tens of thousands of containers and more in a single cluster. Chronos and Storm are other systems that can run on top of Mesos.
In addition to what others replied, you might be interested in this presentation by Benjamin Hindman (who co-created Mesos) to see how it all fits together: https://www.youtube.com/watch?v=F1-UEIG7u5g
CoreOS contains a tool called fleet, which abstracts systemd into a cluster init system. fleet has scheduling rules such as "don't schedule more than 1 foobar on this box" or "schedule this app on a box with metadata region=foo" built in to it.
fleet doesn't aspire to do complex resource scheduling or high density bin-packing. The best way to accomplish these tasks is to use fleet to bootstrap more complex and higher-level services like Mesos, Kubernetes or Deis. fleet will make sure the components of those services are running in the cluster, even if machines die or are partitioned.
Or, more robustly, run Kubernetes, Deis, and Marathon — along with Spark, Hadoop, Cassandra, and so on, all Mesos, all on the same cluster. Mesos is the only technology capable of doing that at any scale.
It'd be very interesting if you know anyone who is doing this sort of thing (fleetd for bootstrapping a more "heavyweight" scheduler) in production, and could prod them to do a write-up on their system. It might inspire some more interesting PaaS products. :)
The current options are $80 development and $200 for production a month. I would like to see a $60 option for starter, but I would imagine the benefits of the mesosphere deployment are better seen on a higher number of instances.
You can customize the cluster ... if you drop down to single core machines you could get a development cluster (1 master, 3 slaves) down to about $40/m I believe.
Will there be Mesos-specific pricing? The full benefit of Mesos abstraction would remove the concept of a physical machine, and allow DigitalOcean to compete with Heroku-like services even better!
Eg, Zookeeper is listening on port 2181 on the public internet?
How is this OK?