I think when you start reimplementing garbage collection in your application, you are probably going down the wrong path and should think about a better way to do things.
Backbone.js is minimal on purpose so that you may implement these types of optimizations. That's why there's many ways to do things in Backbone.js but overall makes the application more efficient.