For those of you wondering what RethinkDB and Docker are:
RethinkDB (http://rethinkdb.com) is "an open-source distributed database built with love. It lets you enjoy an intuitive query language, automatically parallelized queries, and simple administration. Table joins and batteries included. RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn."
Docker (http://docker.io) is "an open-source application container engine. It gives developers a way to package their app and all its dependencies into a portable container which can be deployed on any modern Linux machine, virtualized or not. Containers are completely sandboxed and do not interfere with each other (think “iPhone apps for the server”), have virtually no performance overhead, and can easily be moved across machines and datacenters. Best of all, they don’t depend on any language, framework or packaging system."
I talked to Nick and we looked at what was causing the problems on his server. A few details on the investigation so far.
1. It's not caused by RethinkDB :)
2. It's not caused by cpu, ram or io load. The server is definitely taking a hit, but it still has spare capacity.
3. No apparent issue on the machine itself (a Linode box)
4. It's not a docker stability issue either. The docker daemon is happy as a clam, and the containers that do get deployed also run perfectly smoothly.
5. We suspected that maybe Hipache failed to update its routing configuration fast enough. But that turned out to be a red herring too (unsurprising since Hipache handles vastly larger load at dotcloud).
Our best explanation so far: Docker's default configuration is to assign to itself a /24 ip range. This only allows for 254 distinct addresses, and thus 254 containers (note: the reasoning behind this default was to minimize the footprint on the host system. In retrospect it seems like a silly decision, we'll change it to something larger). Nick then reconfigured docker's interface to a /16, allowing for more containers to be created, and restarted the docker daemon. This causes docker to allocate IPs to new containers from the new, expanded range, while avoiding possible conflicts with the IPs of pre-existing containers. That's where the problem occurs: Docker is doing this correctly, but very slowly, creating a 30 second delay between container creation and successful IP allocation. We're investigating why exactly.
Hi - I'm the author of this site. I'm happy to answer any questions about the technology or implementation!
One of the often asked questions is how to get containers to talk to one another using docker and how to add "runtime configuration." I explore both of these topics and more in the technology piece available at http://nick.stinemat.es#rethink-docker-technology
For those testing Rethink via this service: once you get an instance, click the data explorer link on top, type "r.expr(1).add(1)" (without the quotes) and hit run. Voila - a simple calculator!
RethinkDB (http://rethinkdb.com) is "an open-source distributed database built with love. It lets you enjoy an intuitive query language, automatically parallelized queries, and simple administration. Table joins and batteries included. RethinkDB is built to store JSON documents, and scale to multiple machines with very little effort. It has a pleasant query language that supports really useful queries like table joins and group by, and is easy to setup and learn."
Docker (http://docker.io) is "an open-source application container engine. It gives developers a way to package their app and all its dependencies into a portable container which can be deployed on any modern Linux machine, virtualized or not. Containers are completely sandboxed and do not interfere with each other (think “iPhone apps for the server”), have virtually no performance overhead, and can easily be moved across machines and datacenters. Best of all, they don’t depend on any language, framework or packaging system."