Based on personal experience, I found that paying for the setup, tuning and 24/7 maintenance of a dedicated is way more expensive than using those service.
But I'm one of those crazy people who considers their time and that worth something.
Excuse the snark, but these arguments just keep repeating themselves. There's a reason they're called "services". It's like saying "it's cheaper to slaughter your own cow than to eat at a restaurant".
This is true only if your time is free or you're cutting corners on reliability. For example, using an EC2 medium instance on-demand is $0.120/hour and RDS is $0.160/hour. If you use enough MySQL, that $0.04/hour (in reality, $0.022 at reserved rates) can reach the point where it's worth rolling your own management stack, backups, failover, etc. but there are many people who will never reach the point where they don't have higher-priority investments to make.
+1 for Mongodb on a EC2 instance instead of Dynamodb
We're transitioning off Dynamo as soon as possible as the billing structure makes it either a real headache or very expensive.
For example: if you want to make a backup of a large table or move data around, it'll either take days or cost you an arm and a leg. This is just one of many annoyances. Ever since making the decision to go with Dynamo over Mongo, I've been getting dirty looks from my team.
The set of problems where you're really struggling between mongodb and dynamodb as a choice is practically non-existent. They have hugely different characteristics and vastly different trade-offs. You might want to spend some time reading about the architecture of various databases before just picking one.
1. What kind of EC2 instance are we talking about for MongoDB? If you're not using an SSD, comparing Dynamo and Mongo wouldn't be an apples-to-apples comparison.
2. I buy that the strictly technical costs of these systems is lower, but I'm not certain I buy that the overall cost (including engineer time to set these services up and monitor them) is lower.
1. With a provisioned iops the performance is almost identical to SSD, but yes probably I haven't done a side-to-side technical comparison. I only wanted to point out that I replaced one with the other and I had a good cut in cost.
2. Maybe it is just me, but I really think that some services like mongodb/redis does not require a big amount of system administration work to setup, I have actually wrote many python scripts to automate most of these tasks. but again maybe this depends on the country and the IT culture.
I see what you're getting at, but I'd like to point out that "not a big amount of system administration work" isn't worth money. Have you actually figured out how much it costs you to write many python scripts?
If you have the staff, I definitely can see this. For a small team (in my case, 2) RDS was a no brainer. More than anything, letting Amazon handle backups was a big win. However, in a use case where you need more control (I realize you can ssh/rdp in, but there are limited configuration options) or more performance (the highest performing instance types aren't available) or flexibility (for SQL Server, you can't perform the full range of import, etc activities due to permissions), RDS may not be the best fit.
It depend on the service.
Usually scale is not a problem, but scale in a short time (hours) can be a huge problem.
But most companies would love to have this problem ;-)
So basically we can now replace the painful SQS with hosted Redis! Yayy!
RE: Mongo. If you're using EC2 for MongoDB you're most likely doing it wrong. Not sure if there is much written about the Foursquare problem out there, maybe I'll get around to writing about it, one day. Pretty much the only company that I know off doing bullet-proof, scalable and fast Mongo hosting is http://objectrocket.com/. There's a reason Rackspace acquired them. Unless you're able to get placement of your SSD loaded servers in the same datacenters as Amazon, go with them. It's well worth your time to not muck around with Mongo.
Good on them for doing this, its missing one substantial configuration option though: Redis Authentication. Security groups are great, but it would be comforting to have some challenge when hitting cache.
Well unless you roll with a Micro instance (213mb of RAM and dismal performance), the price for a Small on-demand instance (1.3GB memory) is ~$54 per month. You save quite a bit by going with a reserved instance, where you pay about $70 up-front but then your monthly instance price is almost halved, coming in at around $32 per/mo.
Either way, it's going to be more cost-effective at a lower level to go with something like Redis-to-go, Openredis, etc... if you need something simple.
Long story short, if you're on Heroku and (like me) were thinking/hoping that this might be a nice alternative to the various add-ons available ... I don't think that the juice is worth the squeeze. One, you have to administer and maintain things yourself (which is trivial for something like this, so not a big deal) but the pricing doesn't line up. The Redis providers leading the market today are able to do so because they provision the heavy-hitter EC2 nodes and then chop them up.
It's important to note though you can get a free micro instance for a year on the free usage tier. Micro's are like the section-8 housing of the server world, but free is free and beggars can't be choosers ;)
I got $16.5 per month rate for 1 year of a small (which is 1.7gb).
This is dedicated performance, as opposed to the bad performance of a shared system like openredis, which is $25 a month for a small (200mb). (These systems put artificial limits on things like # of ports, etc. They also share with neighbors as you say, which makes them less reliable in performance - noisy neighbor problem.)
So unless you only need 25mb of cache and have very little load, EC2 is significantly cheaper.
I think the main benefit would be for people already running on EC2/AWS. I would imagine (but don't know) that you'd get better latency and throughput between an EC2 instance an elasticache instance than between an EC2 instance and a hosted redis provider.
Also worth pointing out: if you're running on AWS, you'd get integration with your existing tooling like CloudWatch, VPCs, etc.
And last but not least, Amazon is a mature company that you can be confident will be around another 5 years. I don't have that same level of confidence with the majority of Redis-as-a-service providers.
Compared to normal Redis-Paas, it's on the same line, probably cheaper than the average price on ther market. ElastiCache is basically a way to spin up a EC2-backed instance of Redis, with automatic failover and replication. It only handles one node per cluster, and doesn't handle scaling (you need to manual provision, replicate, switch, etc.).
But if you go with a true clustering solution (like Redis Cloud, www.redis-cloud.com), then you get automatic transparent scaling with zero downtime thanks to dynamic sharding:
Of all the things that you can put to the cloud, I'm not sure why people would want a cloud-hosted memory store. It's easy as heck to set it up in your own server, probably costs less to maintain and host, and faster too as it's closer to your data centers.
If you're using other AWS services, then this makes a lot of sense. It saves you the hassle of setting multiple redis nodes on EC2 and the (admittedly low) maintenance.
They handle automatic failover, which redis does not currently do out of the box. That part is pretty cool. Maybe not worth the price, but definitely useful.
We just spent days moving stuff away from Elasticache due to memcached limitations, and would love to have moved to managed Redis. We're an AWS partner, with the highest level support contract and if we'd known two weeks ago that this was in the pipeline, we would have waited. AWS, why don't you give us any road map for stuff like this?
If you reach out to AWS people, you may get some of this information before it goes public. AWS likes to do private (NDA'd) beta tests of new products with AWS-heavy customers. They work off of the feedback while bigger customers get their needs satisfied.
yeah we recently migrated from memcached on elasticache to redis on EC2 (and bought the reservations), though our migration rolled out about 4-6 weeks ago. oh well, I guess.
According to this it should be quite easy to move your current redis on EC2 back to ElastiCache:
"Seamless Integration: If you are running Redis on EC2, you can transfer its contents to a new Amazon ElastiCache for Redis node. You may also attach a Redis node running on EC2 to an Amazon ElastiCache for Redis node."[1]
Looks to me like this doesn't support more than one redis instance in a cluster (to make a ring) - just replication groups. I'm guessing that's a "TBD" thing. :(
I ended up replacing the following:
- Mongodb on a EC2 instance instead of Dynamodb
- EC2 Mysql instance instead of RDS
- Redis instance instead of Elastic Cache
- Solr on EC2 instead of Cloud Search
- FFmpeg instance instead of Elastic transcoder