I guess if you are super optimizing your website then getting rid of all the useless stuff is fine, but at least I don't notice big enough performance increase to get rid of jQuery even if it's just for the $('selector').