They do this to a certain extent, but the problem here is that the "R" part of "R&D" is not really at issue. While there's a lot of focus in the popular press on "finding" new antibiotics, that part can be addressed by basic research grants. I don't believe we're wanting for proof of concept kinds of things. Unlike with engineering, where once you've got a proof of concept, you can refine and iterate to make it production quality, biology is completely arbitrary.
The issue comes with the "D" part of R&D. That is the expensive part with drugs because you start having to do very tightly controlled animal model studies progressing further into clinical trials with humans. The popular notion that you "discover" a drug is not how things work in reality. Often you discover a whole raft of closely related chemical entities that appear to do what you think they do. You tweak based on further studies in order to improve efficacy and safety. But the system is sort of nondeterministic in the sense that even altering the structure slightly often has knock-on effects you can't predict. Assuming you get something decent, that's what gets made into the an actual medicine. There's a whole branch of applied chemistry that specializes in this called medicinal chemistry. In many cases, you'll read about some promising new antibiotic that has been discovered in an academic lab, but once the chemists take a look at it, you realize there's no way to make a drug out of it. The reasons can be any or all of: the therapeutic dose would be too high, it blows out the liver, it can't be made into pill, the compound is unstable and can't have a good shelf life, it can't be taken orally, etc... Working out all those kinds of problems has always been the realm of pharmas and biotech startups. Academics aren't interested in that stuff because it's not basic research, it's highly applied and is mostly trial and error based on decades of doing this kind of work.
At a macro-level, this is fundamentally what ails the entire pharma sector right now. The development part of making drugs is so expensive and such a complete and utter crap-shoot, that few new drugs of any sort are actually being produced. The last decade has been one of prolonged decline in the industry in terms of truly new drugs and there is no obvious mechanism available to us to make it less so.
