Performance is definitely the biggest problem with fat client applications. As Twitter found out, by crowdsourcing your hardware, you lose a lot of control of the user experience. If you have a 'traditional' web application and it's running slow, it's really easy to by a couple of new servers and the problem is fixed. We can't buy everyone a new computer (sorry!) and force them to use a recent version of Chrome, so it's a challenge.
In the places where we can control the performance, we do that very carefully: CDN loading of all assets, intelligent caching techniques, and of course, finding and removing bottlenecks in the code.
@jarcoal, would you mind letting me know some of your details which might affect performance (which country you're in, what browser, what speed is your computer)? Email me directly at fisher at soundcloud if you'd prefer not to share here. Thanks.