I've worked for one of their (direct) large competitors, but haven't worked for those three companies.
I've currently got active accounts with all three of those VPS providers - I love them, and use them every day - particularly Linode, but also Slicehost/Rackspace, and DigitalOcean. I even have a bare metal server at ServerBeach - which I realize I need to shut down...
At this exact instant I have six terminal windows open across DO/Linode. I host a moderately popular California Food Blog, and have about 15 years experience in various companies that have had hosting responsibilities.
I'm not saying you can't do great things with the VPS providers - I'm just suggesting that the tradeoff between saving $2-$3k (at most) with Digital Ocean, would be more than made up by the technology risk, hassle of having to re-invent a lot of the services that you get automatically from AWS.
That could change sometime in the (near) future - but right now, AWS is an easy (and honestly, all things considered, relatively cheap) solution for this type of application.
But they are just using Amazon's Cloudfront. They aren't using SQS or anything.
What technology risk is there in setting up Varnish and nginx on Digital Ocean? Or better yet some kind of out-of-the-box open source CDN. You would save a lot more than $2-3k.
I'm referring to Digital Ocean's technology. AWS (and Media Temple) had it's teething pains as well in it's first 4-5 years. I remember some fairly broad outages with their back end storage - but they've mostly dealt with those, and, the risk has gone down.
Note - there is another option - Deploy on multiple Platforms and be smart with your DNS balancing (http://www.dnsmadeeasy.com/services/global-traffic-director/) when serving content. Particularly now that Digital Ocean is in Singapore/Amsterdam/NewYork I can think of some useful things I could do with $10/month droplet (2 Terabytes of Transfer) $300/month, in theory, gets me 20 Terabytes in Asia, 20 Terabytes in Europe, 20 Terabytes in North America. Now, whether DO would shut me down if I actually started using that Transfer is another question altogether...
Last test i tested DNS balancing ( from DNSMadeEasy )sometimes doesn't work as well as you did imagined, at least compared to EdgeCast or MaxCDN. Although that is quite long ago.
There is no such thing as an open source CDN; there is a free CDN called Coral Cache (http://www.coralcdn.org/). CDNs cost money because you're dragging content from an origin to edge locations all across the world, and keeping it hot for client requests.
You could simply serve the content out of nginx, but you wouldn't see the performance benefits of keeping your content closest to the end user.
