In case of physics we do not really have access to the later kind, every theory is a current best guess that might at any time turn out to be only approximately correct or under certain circumstances. General relativity is almost certainly not the truth, the universe probably has no singularities hiding behind event horizons, but does it help us to call it wrong?
There is reality and there is series of better and better mathematical models of reality and all of them are wrong in the strict sense until we eventually find the final theory but even then it is not clear that we could even recognize that we have arrived at the destination. In the end a binary characterization as right or wrong does not make too much sense for physical theories, they occupy a continuum of correctness.
I somewhat agree with you, but I think it's very important to distinguish between the two meanings, because different points on the continuum of correctness cause you to come to completely opposite conclusions about the nature of the universe. A bit of quantity leads to a totally different quality. In other words, truth and usefulness are continua, but they are two different and orthogonal continua, and going right on one may mean going left on the other.
My view of the universe is going to be totally different if Newtonian mechanics is the "truth", versus if something like quantum mechanics is the "truth." The former depicts a totally deterministic and knowable-in-principle clockwork universe, whereas the latter has randomness and ignorance that cannot be removed even in principle. Quantum mechanics may not be the whole truth, but it's certainly so different that it shows earlier approaches to be not just incomplete but totally incorrect about their fundamental assumptions, even if they can be used to make useful predictions. Quantum mechanics isn't just more useful, it's more true, whereas Newtonian mechanics might be more useful than some other theory, but actually less true in some sense.
There is reality and there is series of better and better mathematical models of reality and all of them are wrong in the strict sense until we eventually find the final theory but even then it is not clear that we could even recognize that we have arrived at the destination. In the end a binary characterization as right or wrong does not make too much sense for physical theories, they occupy a continuum of correctness.