That problem can't be solvable in any context. There's no way to rule out the possibility that someone else in the world knows your mathematical secret.
Let’s try it this way: what if two people, unlikely as it may be, generate the same key pair? Two people know the private key, and factoring is still hard.