As the number of Rack middleware pieces in a base Rails app grows, it gets slower. MRI and YARV spend a lot more time doing GC as the stack depth grows. Aaron Patterson mentioned that he was working on a modification to the Rack API where each piece of middleware did not call the next piece, reducing the stack depth and drastically improving the runtime performance of Rails apps.
Dropping the rails router, controllers and rack in favor of something like webmachine and resources could potentially speed up rails and reduce the conceptual complexity as well as the stack depth.
I'm curious, have you looked at http://reneerb.com/? I contend as well that Rails is not well suited to REST. As it stands, I think Renee is about 2x faster than Sinatra.
Here's the video where he talks about it (towards the beginning): http://www.youtube.com/watch?v=kWOAHIpmLAI