Inaka has two clients (of many), that are particularly comparable. One used a very "cloudy" Heroku stack with SimpleDB, Ruby, Rails, and Redis. It a number of scalability problems mostly related to the different components not performing predictably under load.
A lot of time spent debugging this custom stack that was supposed to be seamless.
Another client was (boxes running on a hosted server provider),using Erlang, Riak, some Rails, deployed w/Puppet. Comparable data volume; similar "Rails w/Dynamo" architecture, but -- much, much less magic. Much smaller operations bill, less time spent working around cloud platform issues, like inconsistent IO, expensive Amazon charges, lack of visibility into what is happening behind the scenes, etc.
In some cases, Heroku is great; for us, particularly prototyping. But it's not always so clear when it's better to just boot some boxes and deploy some puppet or chef.
From my experience, a single Heroku dyno performs better than a micro EC2 instance (613 RAM) but significantly worse than a small EC2 instance (1.7GB RAM). The first Heroku dyno is free, making it ideal for small sites, but as soon as you want to add anything outside of Heroku's sandbox, the prices are extremely inflated compared to AWS. For instance, adding an SSL cert to a Heroku app is $20. A small EC2 instance costs as little as $17.69 a month with 3 year buy-in (this is the average monthly cost with a 3 year heavy utilization reserved instance). Even if you pay monthly for an on-demand instance ($57.60/m) it's still much cheaper than Heroku considering you can comfortably run several services such as Redis, PostgreSQL, and Resque workers alongside your app. Each of those services will cost extra on Heroku.
Of course, the trade off is the relative ease of deployment. Heroku is the easiest place to deploy, until your app grows more complicated. AWS is 2nd easiest, until you need even more control over hardware configurations. The most difficult deployment is on your own hardware, but it should also be the cheapest option.
Heya--not trying to be disingenuous. I was actually just copy/pasting stats from /proc/cpuinfo. I assumed the audience would realize that everything is virtualized and shared.
I was consistently hitting the 512MB "limit" immediately, even with lightweight processes (not sure how tbh), and I frequently would see processes complaining that they were at 450% or more of their memory usage.
I'm pretty confident that Heroku virtualizes EC2 instances which potentially makes them more resource strapped than they appear.
They do; if you look at Herou's "dedicated" database instances the RAM numbers correlate perfectly with EC2 instance sizes. Even a "dedicated" Heroku database instance runs on shared EC2 hardware, making both its latency and throughput very unpredictable under any kind of serious workload.
Combined with limitations in how their dyno instances work, like the inability to have more than one slug version deployed at once to take advantage of Heroku's routing fabric for hot-deployment, I only recommend Heroku for early-stage systems. Thankfully it's not very hard to migrate off of and still offers an awesome time savings in the early stages - it's just not something to build out on IMO.
dotCloud (http://dotcloud.com) offers the hot-deployment feature you describe with its new "granite" system. It works for an arbitrary number of parallel containers, too, so you can hot-deploy even when scaled out horizontally.
Heroku virtualizes dynos with linux container (lxc) [1]. The good thing is that lxc is very lightweight because it shares the host's kernel, so dynos aren't slowed by another virtualization abstraction in addition to ec2 xen. The bad thing for heroku users is that memory does not need to provisioned upfront for dynos, so spikes in memory usage of neighboring dynos can eat up all the memory in a physical machine.
(Disclaimer: I work at one of the companies in this list).
In the interest of choice and healthy competition (and also in my own selfish interest, per the disclaimer above), I will point out that the arguments in this article apply to all good PaaS providers, of which there are plenty. As an insider in this particular crowd of vendors, I can tell you that the competition is fierce and it is entirely focused on delighting hackers - a trend which everyone here should be excited about.
- cloudfoundry.com/appfog/stackato/tier3 are competing deployments of the cloudfoundry open-source project by VMWare
- phpfog, pagodabox, orchestra: php-only
- engine yard: ruby-only, also the leader in revenue, probably by a wide margin ($28m)
- dotCloud: first multi-language paas, with a bunch of database services as well
- Openshift: previously Makara, now part of Red Hat. Also available open-source
- Djangozoom: django-only
- Nodejitsu, nodester, no.de: nodejs-only
- Cloudbees: Java only
- Azure: solid .Net stack, impressive database service. Didn't use it directly but I've heard it's a great product.
- AppHarbor: an alternate .Net provider
- App Engine: much more restrictive than the others, but super cohesive and a great product if you don't mind the straightjacket
My only real complaint with Heroku is that once you grow out of a basic no frills app, prices really rack up. For example I was looking to move an app I'm currently developing on a VPS to Heroku, but I use a ~200MB Redis database and just that will set me back $110/month.
Heroku is great if you can fit your app within the free tier and great if you're happy to spend a few hundred bucks a month, but in between those two I'm often better off with on or two $20/month 1GB VPS.
I feel exactly the same. I do a few hobby sites and I can't use heroku for this reason. People often reply that it's not meant for hobby sites, but I'll always bet on bottom up technologies, and cutting out hobby sites is not bottom up.
Note my bias: I am the founder of AppFog.com which deploys CloudFoundry to public clouds as a service.
"Heroku is Just Unix"
No. Heroku is not "Just Unix". This statement is completely 100% wrong. To claim that what Heroku or any other PaaS does is "just VPS" or "just Unix" is so far off. People vastly underestimate how difficult it is to run a PaaS and how much technology it takes to pull it off at scale.
I speak from experience, having built one PaaS from scratch and having used CloudFoundry to build a second one. The technology to get "just unix" into a fully managed, n-tier, scalable system is incredibly complex.
I have heard people complain that CloudFoundry is overly complex, but these people judge a piece of open source code that does something that for so long has been hidden behind the curtains. Heroku is at least as complex if not more so, you just never see that part of the equation because you don't have to.
Well, I'll take the blame for this one. I often say this (in a positive light, which I believe is shared by the article) as someone who works at Heroku, and I think I said it to someone else (who had not used Heroku) while in the author's presence; other people might say this as well. The reason I say this is because a lot of people expect a lot of unique APIs or locked-in ways of doing things on Heroku, and it can be quite a while -- if ever -- before they find "heroku run bash" and sort of understand how little magic there is that they have to put in their application.
For some people, it makes the idea of getting on and off the platform less scary. I also happen to like that my local machine models the remote environment in broad strokes quite closely.
Heya! It's extremely awesome to see a comment from you. I've actually been wanting to play around with appfog since I first saw it several months back.
Anyhow--you're totally right. Heroku is definitely not 'just unix'. I really meant this in a different way than it may seem in the article. I was trying to get the point across that there's really no vendor lock-in, you can easily port your code back and fourth from a physical box or vps to Heroku, with almost no delay.
Thanks again! Incredibly awesome to actually talk with you ^^
I think the expression "just unix" is right - at the end of the day platforms like Heroku, CloudFoundry and dotCloud (since we're all plugging our respective platforms) expose an API to spawn and manage unix processes - the way EC2 exposes an API to spawn and manage servers.
It all boils down to the metaphor, and the language that supports it:
PaaS borrows from the Unix metaphor of deployment with primitives like processes, executables, virtual filesystems, sockets and stdout/stderr.
IaaS borrows from the hardware metaphor of deployment with primitives like images, block devices, OS installs, firewalling rules and load-balancers.
The premise of PaaS is that, in a large and growing number of cases, as a developer the Unix metaphor requires less of my time, it maps my day-to-day experience better (As fdr pointed out, the remote platform behaves the same, roughly, as my laptop), and I don't actually need control over the underlying hardware.
So, I'm just a guy who makes video games. A one man operation. I'm mostly interested in interface and game design, I don't know, and don't really want to know anything about running the back end. Just the minimum to get the game running.
Right now I'm running my games on Google App Engine and hosting is about 10% of gross. (Don't know if that's good but it's fine by me). For me, scaling means going from 3 or 4 instances to 20 or 30 in busy times. This all "just works" and I don't have to really do anything.
I've been looking at Heroku because I've been working a little in node and there are some significant advantages to having js on the client and server. (rather than python on GAE)
I was under the impression Heroku would just work too. I may not be an Idiot, but I sure am a n00b.
Is Heroku not for me? Should I stick with App Engine?
Rather focus on optimizing your GAE setup. It's a known entity and it's working for you, now just improve. There are almost always improvements that will reduce costs pretty significantly.
Yeah, but for that price your storage is backed by 7200RPM SATA disks...
If you need speed it will cost you more for 15K SAS disks w/ a sizable battery backed RAID controller, but it will be worth every cent.
You don't want to put too many VMs on them or you'll quickly go through your IO bandwidth.
This is the problem so many devs don't understand. You run out of IOPS for your DB and your app is going nowhere. Managing storage in a reliable, fast and cost effective manner isn't easy.
Consumer SSDs are starting to change that, but you will be still struggling to find many dedicated server offerings backed with SSD storage, and there are still questions about long term reliability (4 years or so) which is typically the period your leaving hardware provisioned for to pay it off.
Consumer SSDs have the problem of not guaranteeing any write durability on power failure, and not providing many write cycles. Never run a database at them. Consumer magnetic disk are very slow but can be made safe with proper care.
RAM buffers work well for large readonly workloads, but write heavy workloads will perform pretty poorly with that kind of configuration.
Mind you its all a compromise, if you know that disk IO isn't a factor for your application then servers like that are ideal. I just won't run a large database on it without something like MySQL Cluster using a Memory table. It would require 2 servers, but REALLY good performance, good reliability (just get 2 dual PSU servers in different cabinets).
Really Heroku is a fine product but your paying a substantial cost for the service to manage some of the systems work. I say some because everyone using services like this should have a strategy to exit should some major event happen (prices raise, service becomes unreliable, company is acquired / folds)
Most of the arguments against running a dedicated server was a straw man argument. YES High Availability is expensive if your talking an app that fits on a single server, but once you can get most services running with a N+1 reliability making it substantially cheaper.
>His harddrive / memory / NIC / etc fails?
Most dedicated hosting providers have an SLA to replace. Rackspace is 1 hour. If this SLA isn't suitable you setup a cluster.
>He accidentally runs OS updates, and breaks versions of his packages?
Welcome to backups... you should have them. Yes its expensive, yes its a pain in the arse... but you should have them anyway even if your on heroku. Lets be professionals here.
> He wants to instantly rollback to a previous release of his codebase?
Welcome to code deployment tools... Just because your not on heroku doesn't mean you can't have them
>He wants to add another server to handle incoming HTTP requests?
Its called resource planning... You know what those pretty little graphs are for. Again you should be doing this regardless of the service you are on.
> He needs to spin up a database read slave to handle a high amount of read requests?
See capacity planning.
> His load balancer fails?
Fail to wire modules, Clustered load balancers, virtual load balancers on a cluster... many options
> His Fabric script stops working because he renamed his project or reorganizes his application directory tree?
Testing... read about it... It really helps. You can also mess up any codebase by messing with the directory structure and not testing.
> Or one of an infinite amount of other possible problems?
All of that is obviously possible (wonderful even!) to learn. But the thing is, it takes time. And on a small team, you really have to pick your battles.
I don't know shit about load balancers. It's a hole in my knowledge. Eventually, I'm sure I'll get there. But I'm really glad I don't have to learn it right now. Because honestly I have my hands full putting out other fires without having to think about it.
So you've never done any systems engineering or architecture, apart from the heroku mold (nail <= hammer)... Given that, don't you think your post was a teeny bit venomous?
What people are waiting for is a set of tools that lets them run their own "mini Heroku" across multiple IaaS providers.
I should be able to glue together my own cloud-based load balancer, deploying my app to webservers on different continents, with a monitored master-slave database set up -- all with freely available open-source software tools.
OpenShift is Redhat's open sorce PaaS. Ruby, node, python, java, php, etc. They run a free service on top of AWS, and you can deploy it on top of AWS, rackspace, hp, or anywhere on top of fedora or RHEL 6.2.
I looked at OpenShifts codebase and I would throw this away after 15 minutes of code review. Looks like they wrote it in Rails because it was cool at the time, but had no idea how to build Rails apps.
Would you mind adding more specifics? I'm curious about what bad Rails practices you found in the OpenShift codebase. (I've never worked on it, used it, or looked at the code, so my question comes from simple curiosity.)
You should have a look at Stackato (http://www.activestate.com/stackato). While it is Cloud Foundry based, it is a fork with significant improvements, including secure containers, integrated web console useful to both users and admins, user/group delineation, and ... Heroku buildpack support. Heroku-in-a-box, you run it where you want, on your terms.
We make heavy use of Cloud Foundry, and have modified existing services and written an entirely new on and integrated it all via the included bus system. It seems like a logical enough design (though the code can be messy at times, it's alright).
I'm curious what parts seemed overblown or complex?
Sorry -- I assumed too much knowledge. Cloud Foundry is really not at the point where it can replace Heroku. Its 80% of the way there, but every needs a different part of the last few % so you can expect to have to add to it based on what you need.
> Cloud Foundry is really not at the point where it can replace Heroku.
I'm curious what Heroku still does that CF can't.
As another person who is building infrastructure on Cloud Foundry like Lucas, I am biased, and don't quite agree with you.
I think from a user perspective, you can do as much with CF as Heroku.
Architecture wise, you can't compare CF to Heroku because its closed source. Heroku could be even more overblown and complex than CF for all you know. And even then, a PaaS by nature is sort of complex.
Administration wise, CF still has a ways to go. It isn't the easiest to deploy and manage, docs need work, etc. But that isn't something that is user facing or comparable to Heroku.
Heroku gets you into big trouble when their abstraction starts leaking and you need to dig deeper. If you never get to that point, this article makes sense, but when you turn that corner, expect a lot of pain.
Wow, this is awesome, so I must be an idiot. I've been playing around with Heroku with a simple NodeJS app that I don't care about the performance of for the last week. I've been scouring their website for information exactly like this and I couldn't find anything this comprehensive. Thanks so much for figuring this stuff out. Just answering the simple question of "Is the 750hr's free for each app or is it per account?" is golden.
Anyone have an example of a simple NodeJS worker dyno? The web one is easy, now I'd like to see the other. I've been searching around and haven't found any good examples. Thx.
I didn't like it that much when I spinned 3 dynos and was hoping that when the 750hrs would be consumed, the service would be disabled.
Worse, heroku asked for credit card info for account validation, and I never expected that I would be billed when using the free tier service, even when spinning extra dynos. I opened up a ticket and they said that they would look to improve this some point ahead.
Luckily, I only use "virtual" credit cards (specially for "validation" purposes) and mine was expired before I was billed.
This is one of the things I actually quite enjoy about Heroku--not to mention that with their new heroku-postgresql:dev plan, you can have up to 30 (yes, 30!) free databases PER APP!
You can have like, 5 masters, with 5 readslaves each, etc. It's super awesome.
First, please don't abuse the free databases. Second, the free "dev" plan does not support fork or follow, nor do we plan to support those features on that plan in the near future.
"Yah--it isn't very hard to spin up some simple infrastructure services using Chef--pretty much anyone with a couple hours of free time can figure that out."
This person has obviously never touched Chef. Chef's learning curve is steeeeeep.
I have my own server that I keep in a colo for a reasonable fee; I run small sites for family members and my email off of it. However for things that actually get traffic I go to Heroku or AWS, I don't need the headaches that come with a traffic spike on my server it's just not worth the hassle.
We make a similar product to Heroku: Continuous Integration as a service (https://circleci.com). I'm amazed at how many developers think their time is free.
"I can just set up Jenkins" is the CI equivalent of the behaviour the OP rants against. And you're right, if your time is free. But it isn't free, and just like Heroku handles your security, scalability and lets you get back to work, so too does hosted CI handle machine setup, test speed, optimization and parallelization, without having to manage it all yourself.
Long story short, developers significantly undervalue their time (and sometimes their bosses do too).
A lot of time spent debugging this custom stack that was supposed to be seamless.
Another client was (boxes running on a hosted server provider),using Erlang, Riak, some Rails, deployed w/Puppet. Comparable data volume; similar "Rails w/Dynamo" architecture, but -- much, much less magic. Much smaller operations bill, less time spent working around cloud platform issues, like inconsistent IO, expensive Amazon charges, lack of visibility into what is happening behind the scenes, etc.
In some cases, Heroku is great; for us, particularly prototyping. But it's not always so clear when it's better to just boot some boxes and deploy some puppet or chef.