Penrose's 'Road to Reality' [1] is a kind primer on where the math comes from, as it applies to physics. Kind of a philosophical walkthrough of how math applies to physics. It is nowhere near as concise as Feynman's lectures, but it does complement them pretty well, while getting more into the math, and why the math is needed to describe various aspects of physical reality.
As it happens I bought both Road to Reality and Lectures on Physics at the same time, about 14 year ago. I read and re-read Lectures on Physics but I was never able to finish Road to Reality. I have kept my aging copy and hope to one day get through it, but it's a MUCH more difficult read, at least in my opinion.
It definitely is quite difficult, but it's also very inspiring in finding various topics to read more. I must have bought at least 20 books from the vast bibliography in the appendix of the book.
I have a love-hate relationship with that book. It elegantly shows how to think about the topics, but it also left me with many unanswered questions to which I could never find answers with any amount of Internet searching (even though those answers are known to the humankind, buried in a set of complex topic-specific books).
Feynman's Lectures are much more complete in that sense, even though as other comments on this thread note, the reader may not be able to use the learning to solve practical problems without going beyond Feynman's lectures.
[1] https://www.math.columbia.edu/~woit/wordpress/?p=154