Are you referring to his quote "If all of mathematics disappeared, physics would be set back by exactly one week." or his more general belief of if you can’t explain something in simple terms, you don’t understand it?
Maybe the issue is that some things are too complicated to be explained without maths and aren't accessible to lay people as they're so counter-intuitive.
Mostly the latter. Doing the maths doesn’t always mean understanding something. As a simple example it’s pretty easy to treat gravitation of a spherical object mathematically as a point source, but it was quite a conceptual leap, and it doesn’t mean you understand why you can treat it as a point source (and when you can’t).
I’m also reminded on the scene in “Severance” where the members of MDR are unable to explain their job to another department.
It's strange as I typically think the opposite - you don't really understand something until you can measure and quantify it. e.g. we don't have a handle on consciousness yet as we can't reliably measure it.
However, when it comes to a "deep" understanding, then maybe numbers aren't enough and we need to be able to intuit what's going on that creates those particular numbers.
