There are no closed form solutions to uncomputable problems. You can encode them in math. You can also encode them in a program. The result is the same.
In fact, the absolute limit of computability is so interesting that people have explored it in the abstract sense. For reference, see busy beaver numbers on Wikipedia or Scott aaronson’s awesome essay on finding the bigger number.
> But surely there are uncomputable objects which you can make mathematical statements about.
Absolutely, one famous example is the halting function that has type (program description, program input) returns boolean.
A symbolic formal system can "reason about reason" and manipulate descriptions of objects that are themselves uncomputable, countably infinte, or even uncountably infinite.
Not a particularly interesting proof.