The hammer analogy doesn't make much sense because for a hammer we can actually use our scientific knowledge to compute the best possible way to hold the tool, and we can make instruments that are better than hammers, like pneumatic hammers, pile drivers, etc.
With your argument, we would be stuck with the good old, but basic hammer for the rest of time.
That seems like a different analogy; making better hammers is a different thing than understanding why holding a hammer a certain way works well. We did eventually invent enough physics to understand why we hold hammers where we do, but we got really far just experimenting without first principles. And even if we use first principles, we are going to discover a lot more by actually using the modified hand-held hammer and testing it, than necessarily hitting it out of the park with great physical modeling of the hammer and the biomechanics of the human body.
And in any case, I'm not saying we shouldn't search for deep understanding of what hyperparameters work on a first try, I'm just saying there's a good chance that even if the principles are fully discovered, it may be that calculating using those principles is more expensive than a bunch of experimentation and won't matter in the end.
That's the trick about science, it's more about finding the right question to answer than how to find answers, and often times the best questions only become apparent afterwards.
With your argument, we would be stuck with the good old, but basic hammer for the rest of time.