Personally (I would probably get shot by saying this) but responsive design is a real problem, it is not a feature, but its a bug - a horrible one.
By appeasing to many screen sizes, we create worse UI for all 3 sizes because no one really designs them for each size separately. Widths are in percentages, flex layout allos collapsable columns and the whole thing is not built from ground up. Either the designer starts with Desktop first and then mobile is a second thought (Bootstrap), or all class names are mobile first (Tachyons).
If we design UI from scratch for each of the 3 or 4 classes of screen sizes, it would be so much better. Completely, from scratch. Not taking the Desktop layout and collapsing the columns. But doing things like button sizes should be smaller for Desktop (mouse) and larger for mobile (finger). Not a single framework does this.
Overall, I agree it's a problem, but I see responsive design as a (perhaps poor) solution to the convergence of three more fundamental problems.
The first being screen size fragmentation. Most tablets fall into a similar size/scale category, so you can have buffers (kinda like how you were saying). However, there are still corner cases where (e.g.) a tablet in landscape mode happens to trigger a different size category. The fact that resolution helps in fingerprinting users is pretty indicative that fragmentation is a huge problem.
The second being the lack of a good way for a page to know exactly what type of device it's displayed on. There are hints/etc. to get close enough. However, it's still a fundamental truth of web development that the browser can always lie to you. Not necessarily a bad thing, but a design challenge regardless.
The third is less of a technical or even a design problem, but I personally think it's the worst. More and more, companies are making increasingly complex sites with no real thought to user implications. I believe this will always be true in a web where the real customers are ad companies and search engines and the user is the product.
On the lines of another point you've made, I'm not suggesting we give up; there are still plenty of ways we could and should improve designs and frameworks (you've outlined some really good ones).
I'll admit I'm not up to speed with a lot of CSS, myself. I'll definitely be looking into how to better use columns.
While I think frameworks can be helpful to keeping a standard look and feel to the web, I would definitely agree that they're often used as a crutch.