Hey everybody, Solomon from team Docker. We're all lurking here and are happy to answer questions.
Just so you know I conzider Hacker News to be a key part in Docker's success. Your feedback, both positive and negative, made us feel like we weren't crazy and were building something that mattered. I hope in the end you will feel we live up to the expectations :)
We have lots of really cool things in store this year, I hope you will like them.
Happy hacking on behalf of the entire Docker team and contributors.
Hi Solomon. Congrats on the round. We're building on top of Docker a reproducable genomics platform at Seven Bridges Genomics. For us, Docker makes science and healthcare reproducible. Let's brainstorm on how we can bring wider visibility to Docker in that vertical :)
I'm working in the genomics space as well. We have an open source interface for filtering and examining variants for clinical diagnostics called Varify. We presented a poster at ASHG and are trying to get the open source code we have out on github documented and installable (https://github.com/cbmi/varify). My experience is that most of the analysis companies have the final variant display, filtration, and reporting as a gap in their capabilities and don't have the time or resources to devote to it 100%. Clinical diagnostics (where we are focused) has very different usage scenarios from discovery work, so it may not be a good fit depending on your focus.
Regardless, I'd love to hear about your experience with Docker and NGS pipeline work. Looked at your profile but there was no email. Shoot me a note if you want to chat (email is in my profile).
From what I understand, Docker looks like it can solve a lot of the practical issues involved with a Service Oriented Architecture (SOA).
Have you guys looked at targeting the enterprise market? Lots of enterprises that I'm talking to are looking into building their own private clouds for their SOA, and Docker could make that much easier.
From what I've seen on the technical side you just need rock-solid stability (or a strategy to achieve that via distributed architecture) and good management tools. Meat-space requires more work/investment (support, training, certification etc).
Congratulations. I'm glad to hear that HN feedback was a factor in making the team feel they were building something that mattered. I know there were always plenty of comments in threads about Docker that were dismissive, but count me among the people that were always pleased to see containerization on linux made trivial.
Given that Docker is licensed under Apache 2.0 which allows the venture backed Docker Corp. to create a closed proprietary version of the software... will outside contributors be compensated?
This is Ben, the CEO of Docker, and Solomon's partner in crime. Being based on Apache 2.0 (and adopting an open design process, having a large number of non-Docker Inc. maintainers, having over 300 contributors) should be read as a strong signal that we will not go proprietary or even "open core." Anyone can use and modify Docker under Apache 2.0 without fear of "copyleft" or similar restrictions, which creates a strong disincentive for us to abuse our position by going proprietary. Our business plans, as stated in the blog, are around commercial support and hosted services. (In addition, being Apache vs. GPL makes it much easier for enterprises to adopt us, and much easier to get integrated into projects like OpenStack.)
No :) If we released a closed-source version we would get forked in a instant. It would be an extremely stupid thing for us to do.
We've stated repeatedly that Docker inc. will make money by 1) offering support and professional services to enterprise customers, primarily through channel partners like Red Hat and others; and 2) offering hosted managed services for things like image hosting, orchestration, container tracking etc. Since we have been operating a PaaS at large scale for several years now, we know how to sell hosted services and it makes sense for us to focus on that.
"While Docker itself will always remain free and open source, we are busy gearing up for the launch of both hosted services around Docker and commercial support for Docker."
You have it backwards. If it was GPL, that would prevent other people from creating closed proprietary versions of their software. GPL or AGPL would prevent competition. Since they are the copyright owner they get unrestricted use to the software even if the rest of us had to follow the GPL.
Awesome work. Quick question, how closely do you guys plan on working with (or if at all) the CoreOS guys? I'm really seeing the future of all servers running ubiquitously on CoreOS + Docker setups. Thoughts?
No thanks. I would prefer that Docker work equally well on as many base operating systems as possible. Having any form of coupling would turn Docker into another locked in platform which is exactly the opposite of the way I see the future of deployments.
I mean seriously you really want a single vendor owning the base OS of all servers ?
I'm working on an itty bittier jeos vm that allows basically any linux userland via docker (rhel, deb, arch etc). It'll ship with zfs and grsecurity on a long-term stable kernel. Completely open source, hackable (core src packages similar to arch PKGBUILDs) and rebuildable from source.
During our alpha period, Chaos Monkey (i.e. random reboots) is built in and will give you plenty of opportunities to test out systemd.[1]
Great. While I understand why they do this, I don't want to have to setup a cluster with automatic failover just to try it out.
And then there is etcd. I'd love to use etcd - all I want is a minimal Docker image with etcd installed. When I looked, it seemed like I had to either build it myself, or use a CoreOS image.
I know it's alpha software, but that isn't a great experience.
Boot2Docker[2] OTOH: one 24 M image, burn it to a USB stick, boot it (in about 5 seconds!) and now you have the perfect Dockerhost.
To answer your question, our priority is to make sure docker works great on every distro, large or small, old or new. As some of the answers below will show you, sysadmin-world is quite a fragmented place, and I don't think we should expect "one distro to rule them all" situation. Instead I expect we will see more and more niche players solving a specific set of problems for a specific group of sysadmins, catering to certain tools preferences (openstack, mesos, hadoop/yarn, cloud foundry, juju, systemd, and so on). I think CoreOS is showing the way for a lot of those future distros. We want to make sure docker can be a good citizen in all of them.
My entire focus at Docker is building great relationships with the companies building software that use Docker as a part of their stack, whether it be for CI/CD purposes all the way down to building tools like PaaS.
So far we have been incredibly grateful for all of the interest and all of the amazing projects that are out there delivering real value with Docker today. The ecosystem continues to surprise and delight.
Please get in touch (nick@docker.com) if this is something you're interested in exploring. I'm also hiring if these are conversations you'd like to be having with others.
Do you guys have any plans to give BSD systems (FreeBSD has Jails perfectly suited for Docker) some love, given the amount of money you were able to get? :)
Yes :) We are working on pluggable execution drivers. The current driver (lxc) will be replaceable by eg. libvirt, plain chroot and eventually zones, jails etc.
There is actually someone experimenting with Jails support already (can't remember his irc handle atm). If you want to come by irc or open a github issue, we can put you in touch.
I've wanted to build something like Docker for FreeBSD Jails for a long time. I've been watching the Docker project since it was announced, and I spent this past week almost exclusively working with Docker and testing it for a possible production deployment (hope you get have a 'production ready' release soon :). In short, I would love to have Docker on FreeBSD!
Not to take away from Docker's achievements (seriously, well done), but it's a funny world we live in, Docker gets 15M (or "another startup casually gets $XXM), while OpenBSD foundation (which pays for stuff all of us rely on every day) have to beg for donations and 100k is a huge achievement.
I feel similarly. Docker's founders deserve only congratulations, they're doing a cool thing and asking for money for it. And Linux's terrible and primitive containerization technologies are finally growing up to be able to match Solaris and FreeBSD. But I don't find "funny" strong enough a word when OpenBSD, which is IMHO a far more fundamental and important technology than containerization and should be easier to monetize than Docker, is struggling to survive.
I think it comes down to the personalities of the leadership, and where they want to take the projects. But it's a crying shame that OpenBSD is struggling. Not related at all to Docker, but a shame none the less.
I can give 100k to docker and expect to get back 200k.
If I give that money to OpenBSD I'm expecting a general "the world may be better" and a loss of 100k.
While a nice thing, this is not what I'd want people managing my money to do.
I think that with the proper service infrastructure on top of their open source development model, OpenBSD could also bring in 200k on a 100k investment. But OpenBSD would have to be pushed and marketed as Docker has been, and the service side organized as a for-profit organization.
I'm really glad you brought this up! This topic is actually very relevant to what we're trying to do with Docker.
Think of it this way: we (the creators of an open-source project) have built for our project an infrastructure capable of supporting its long-term development and success without falling victim to the "openbsd effect". We hijacked for ourselves a venture-backed company and are experimenting with a symbiotic relationship where both the project and the business win without corrupting each other. We get financial, legal and marketing "artillery" in the form of a corporate structure which is dedicated to the success of the project without distorting it. We have large stakeholder (me) who is also the maintainer of the open-source project. We have a CEO (Ben Golub) with a track record of incredible success in general, of successfully managing an open-source company in particular and with a reputation of doing business in an honest and trustworthy way. We have investors with a track record of being patient, seeing the big picture and backing open-source companies and their creators in a sincere way. These guys could have fired me at any point of our transition to this model, and didn't.
So, sure, we got money "casually" (I'll leave that battle for another day), and OpenBSD struggled to get basic funding. And obviously, OpenBSD and Docker are not in the same league as open-source projects (although I hope one day they will be).
But, respectfully, I think this is more than "another startup gets $XXM". We are experimenting with a new model for the funding of open-source. One where the original innovators get to capture some of the value they created, fueling even more innovation in a virtuous circle of "innovation -> better product -> more money -> more innovation".
To me personally, were we to succeed, that would be a welcome change from the current situation, where the initial inventors are trampled by savvier entrepreneurs, who race to the local maximum of the most profitable derivative business model with the highest barriers to entry, and proceed to not reinvest their profits in the original invention, thwarting the virtuous circle. For example, Xensource produced the open-source hypervisor and we got Amazon EC2. Linus produced Git and we got Github. I think these products are great and useful - but they are a local maximum. With the raw material of git, hypervisors and a commodity open-source operating system, we could have produced so much more aggregate wealth!
At Docker we want to be both git and github. We're open-source people first, but we know how to sell a saas product and run it at scale. We know how to raise money and build a profitable business with that money. And that profit gets directly reinvested in the original innovation - Docker.
And I think that's the future of open-source: if we as a community want to see less "OpenBSD episodes", we're going to have to roll up our sleeves and go about building real business value on top of our inventions. Otherwise others will, and we'll only have ourselves to blame.
Shykes, I understand how I sounded, but I meant absolutely no offence to your achievement as a team. You absolutely deserve the money and even in my little world you're not "just another startup" as I'm a huge fan of docker and believe it's the future.
"Casually" is more of an observation of HN, as not a day goes by without someone getting millions in funding, but I'm aware of how not casual the whole approach to getting that much bank is and how much work is actually involved.
docker does in fact live in the real world and play by its rules, it's not a bad thing, objectively, only an idiot wouldn't. If you play your cards right, you'll make money, won't ever need to beg for donations and will do just fine becoming the new standard.
OpenBSD tries to live in an idea, which is close to my heart and I imagine close to anyone, who grew up toying with kernels and working on distros. The recent donation thing was like a reminder that Santa's not real and there is no magical fairy land, where just putting your head down and doing what you believe is the right thing for the benefit of everyone, without thinking about money, will be rewarded.
No offense taken whatsoever. I agree it's an important and unsolved topic, and I myself still have a few old OpenBSD posters from my teenage years :)
Having grown up in a family of "starving artists" like they say, I see a lot of parallels between open-source development and, say, music, painting, dancing etc. There are many people who produce art as a "gift" to society, without financial motives. This almost always comes with a tension between the quality and intensity of your work and how you will earn a living. Perhaps you have the luxury of not requiring payment for your work, because you have another source of income - but then you might be limited in the time and energy you can invest in your art. Or conversely, you wish to focus your entire time and energy on your work, but find out that securing funding from your artistic work (whether royalties, donations, salary or grants) quickly becomes a job of its own - it is common for artists to feel that the grind of securing money for their work has become their primary job. Some discover they have a knack for it, or just get lucky, and find immense commercial success. There is a commonly held notion that commercial success is inversely proportional to intrinsic artistic value. But of course it's hard to know for sure since that is such a subjective thing to measure. Is Justin Bieber's art "less good" than the local rock band I watched last night? It depends who you ask :)
So why is it difficult for artists, sometimes immensely talented and critically appreciated artists, to secure funding? Obviously a complex and immense question. In some societies the notion of paying someone for art is inexistant, because everyone produces art on a daily basis as part of their culture. In our society, it is accepted to pay for art as a service, but only to a point, and following rules that can be hard to decipher because they rely on guessing other people's emotions, like fashion or the capital markets.
At the end of the day, I think part of the problem is that art, like writing open-source code, is an enjoyable activity. People like to do it. Many people turn out to be good at it. Some people even pay for the privilege of doing it. Which means it will always be a buyer's market.
I'll be watching your adventure with keen interest.
I'm also building an open source project that I want to avoid the "openbsd effect" and sustain itself, more like the Linux kernel.
I have a company that I'm using as a vehicle to bootstrap this project. The company sells services and acts as an example for others to start offering similar services (ideally including Redhat etc one day). To play it safe I don't allow the company to have any intellectual property and I don't sell any equity.
The idea of the company as a vehicle rather than an asset has been a powerful one for me.
Happy to chat about it anytime, and please let me know if I can help!
It should go without saying that I don't think this is the only route for supporting open-source. Linux comes to mind as perhaps the most successful open-source project of all time, and it is backed by a non-profit foundation funded by member companies. (for example Docker inc. is a small member of the foundation, so a small fraction of the $15m we raised will pay for a small fraction of the salary of a core linux maintainer).
> We hijacked for ourselves a venture-backed company and are experimenting with a symbiotic relationship where both the project and the business win without corrupting each other.
Well, 15M gives investors control, and (even out over several investments) reasonable likelihood of profit. But, then again, you point out a great idea: to somehow tie large investments and open source projets. I'm not sure entrepreneurs who struggle getting funding would want to had one more stone to their neck, but it would be great PR to have either a repeated entrepreneur, or an accelerator, lock x% of the funds they raise to donation.
Without inside information you have no idea if this is true or not. It may very well not be. Facebook raised over 2 Billion prior to their IPO and never gave up control. $XXm Investment != Control.
100k was a gift with no strings attached and 15M is an investment with planned profits, control etc.
By the way, today's docker user base and importance isn't that small comparing to OpenBSD. At the moment there's 377 people on #openbsd channel and 715 people on #docker channel (freenode).
Very true. Do you think we could take the Gittip model a few steps farther, and have companies use it to fund open source projects they are built on (like OpenBSD)?
Sure, big players like Google contribute to open source projects, but I'm thinking along the lines of smaller companies pledging 50k/year because they rely heavily on ___ open source technology.
I'm not sure that I understand your point -- people place a value on things but that doesn't mean anything that makes less money is any less important.
That's exciting. I looked at Docker briefly, and it has the "Ooh, cool project, can envision the potential" bit in spades but unfortunately is at that unhappy stage of maturity where e.g. betting AR's future on it not blowing up or shutting down would be a bit more risky than the benefits currently warrant. I'm really looking forward to when it's production-ready.
In addition to the funding, we have bet on a Linux-style development model where the project is not just open source but open design: ownership is federated across key individuals with strong ownership over particular subsystems, with very low barriers to entry and a diversity of employers.
This is one of the reasons Red Hat felt comfortable betting on it so quickly - and you should expect other major industry players to folluw suit this year :)
That's awesome, and I can respect that you're trying to tell me "We are trying to make Docker the project survivable even in the event of Docker, Inc ceasing to operate as a going concern.", and I have much less stringent requirements for maturity than the Fortune 500, but...
As one geek to another, the thing which stops me from Dockering up my entire infrastructure in February is that: of that NxN compatibility matrix that Docker will some day liberate me from, I feel like I'm going to be the first production system in a lot of the cells, I will have to replace my existing cobbled-together infrastructure which I know to be borderline functional with hand-rolled work to Dockerize it, and I figure that when there are inevitably bugs I will be the one to weep tears of blood about them.
Coming from the ex-CEO of a 24/7 managed hosted service... I absolutely hear you!
So, we'll simply work our buts off to make the risk/reward more and more appealing for you... And in the meantime we would love to go over your particular stack and tell you how far off we are. Apparently we can afford a team of solutions engineers to do that now :)
If you're interested feel free to ping me at solomon@docker.com. I would be honored to explore your cobbled-together infrastructure!
I know the feel about taking some unnecessary risk, but still you will certainly not be the first to try to clean the floor:
A lot of people are actively using Docker right now. For instance I'm building https://reesd.com on it (i.e. I don't have a prior system that I replace). Getting a few bugs from Docker is not different to me than getting a few bugs from my own systems. I write my integration tests and move along.
Other people are further along the road building a lot of different things, trying a lot of different ideas. I would think that unless you're doing something radically different, a lot of ground has already been covered (and will be covered until you have completely switched to Docker).
Another reason I was not afraid of Docker is that it is built around existing pieces. For instance some people say "Docker is just a script around the lxc tools, what's the point". That, in my opinion, is reassuring. If I need to, I will be able to drop to the next layer, or another similar project will take over.
I'm in the same boat: I think this has huge potential but with my skill set it would not be wise for my company to be a trailblazer on ops.
@shykes: What would convince me is one or two blog posts from respected members of the community (patio11 comes to mind :-) where they detail a successful transition to use Docker in their infrastructure.
This is something we're working on. Right now Docker is successful in projects and companies who want to be on the ground floor of revolutionary product and have the capacity to invest in / contribute time to dealing with maturing technology.
Not all companies have the benefit of being in such a position, and we're working hard to make Docker adoption a possibility there by offering Services (through partners) and Support (through us) around it.
We will not be the ones to make the judgement of when we've hit that level of maturity, but we will be publishing use cases / white papers / details about people who believe we have.
What I would encourage you to do is to start thinking about it and take advantage of the fact that we have a vested interest in making your project successful. Get in touch, tell us about your needs, and help make Docker better by making sure your use case is on track for being rock solid.
I've been watching this Docker excitement with interest, and went through the little demo thing on their website (which was quite cool!), but still have trouble connecting the dots on how exactly I'd use it.
I run your standard Heroku/PG/RoR web-stack app. Is anyone else using Docker in that environment? I was vaguely thinking we could set up a Vagrant + Docker combo for our development team to manage dependencies of gems and native apps and whatnot. Is that how Docker is used? Would getting set up on Docker now make an eventual transition to, say, AWS or self-hosting later easier?
I'd love to hear examples from how people with a similar tech stack are using Docker.
Yeah, the Vagrant + Docker use case is how I'm using it. This is the flow that I'm pushing my company to use:
* CoreOS/Vagrant
* Docker containers with basic infrastructure (eg LAMP/Ruby+Postgres)
* Containers 'forked' for each client (and company project)
* Central container repo for the team to use
As soon as CoreOS is ready for production we're going to look at using it for deployment.
We might not be using it 100% correctly, but so far it's nothing short of amazing. The basic 'git for OS' flow I get out if it makes me smile every time. Dockerfiles are way, way, way easier than Chef, Puppet or Ansibke, and the central repo is a synch to picture and imagine in comparison.
So, you're not running Docker in production at this point?
Are you putting the whole stack in one big container and running one container per Vagrant VM? Or are you spreading a client stack across multiple containers?
Congrats Docker team. Docker is an amazing product and I've been following it closely as well as playing with the latest version. Not one bit of the hype is vaporware. Cant wait for it to get a little more production ready. I'm sure anyone who has done even a small amount of dev-ops can appreciate what a huge pain point docker will solve. VM's have been commoditized, but with docker, they can be obliterated altogether.
One question I have for Solomon -- do you guys have a product roadmap on the site anywhere ? As in, what features are planned etc. and by what release/time ?
Yes, we are actually working on exposing our high-level roadmap publicly. We are currently testing a public Trello board, but the choice of tool is not yet final.
All of this is being discussed openly on the #docker and #docker-dev IRC channels on Freenode. Feel free to jump in and say hi!
A friend of mine recently asked me "I'm looking for a job. If my only criterion (it isn't) was which companies are most likely to 10x in value in a few years, which companies would those be?"
Though not quite the same, IIS application pool isolation (separate app pool and worker process for each website - which is the default from IIS7+) gets you a bit of the way there. This has improved security, performance and reliability massively for us.
But yes, speaking as a shared webhoster devops kinda guy, I agree, it would be nice to have containers on Windows too.
The Apache 2 license is great and using a DCO sounds great. Is the project Governance documented somewhere?
Congrats on the amazing achievements so far!
thanks but I was more hoping for something describing the process used to decide if/how/when to incorporate pull requests, roadmap, software quality before releasing, ... Do maintainers discuss to reach consensus, do they vote, something else?
It could be that this is all Docker Inc's prerogative, which is fine. I just don't see it spelled out.
And in general the rest of that hack/ directory. We use a process that is completely open and not at all limited to Docker, inc. Several core maintainers (and countless contributors) are not employed at Docker inc.
Generally speaking the reference for how we organize is Linux.
So... I'm quite sleep deprived due to a sick kid (nothing serious) and haven't really followed Docker much in any case.
Why would I "Dockerize" my infrastructure, and what benefits and challenges does that entail? I mostly do fairly simple RoR web apps that run on something like Linode.
That's not a very helpful reply, really, as a lot of the first hits on Google are kind of vague, and I have no idea whether the whole thing really applies to what I do in any case.
Just so you know I conzider Hacker News to be a key part in Docker's success. Your feedback, both positive and negative, made us feel like we weren't crazy and were building something that mattered. I hope in the end you will feel we live up to the expectations :)
We have lots of really cool things in store this year, I hope you will like them.
Happy hacking on behalf of the entire Docker team and contributors.