There is most certainly a massive causal chain between physics and what humans call "math", because everything humans do is determined by the laws of physics. The causal chain leads through millions of years of evolution and tens of thousands of years of culture. "The numbers 1 and 2" are a complex web of analogies that have not actually been demonstrated to "exist" outside of our minds, so evidence that alien mathematicians would have words for them is much weaker than our intuition would suggest. The question that must be answered is "Given the constraint of precisely modelling the world in a useful way, to what extent are all rule-based systems isomorphic?"

Responding to your second point, I'm afraid I can't agree with you that unfalsifiable propositions are "useful" discussion contributions - especially not with an "open mind". The only criterion on which such propositions can be judged is whether they are fun to believe, and that is a very dangerous muscle to flex.