Most decent rack servers will have some out of band management to allow this. If that fails then some dude at Equinix rolls over some stone age KVM trolley for you for a few hundred dollars..
Plugging those things into the public Internet is a recipe for disaster, and not at all worth it if you just want to use the remote console. They are a lot more than just "KVM over IP", they own the machine totally and completely. BMCs can generally load firmware into CPUs (and NICs, storage controllers, etc.), they can read/write arbitrary locations in host memory, etc.
Those are for plugging into an air-gapped management LAN; their utility is pretty much nil in a shared datacenter. (Which is implied by colocation.) - I would not even connect IPMI unless I "owned" enough of the rack to justify having my own router and switch. If you just have one or a handful of machines, cabled into whatever top of rack switch the DC provisioned, your security posture is way better off just using their crash cart. (Which is roughly the same magnitude of risk as having a keylogger installed; many many magnitudes less than every secret on your machine being compromised.)
This is why I have a problem with Supermicro. They used to have IMPI that automatically shared the motherboard's primary ethernet when nothing was plugged in to the IPMI port.
You can't disable that in the BIOS. You can't disable IPMI in general, and you can't modify any of the IPMI's settings aside from IP, nor any credentials. They expect you to have an old Windows installation with Java installed in an old version of Internet Explorer to configure the IPMI so that you won't get owned.
Ok. That's bad, but it gets worse. There's no way to disable it via jumper or in hardware in general. In essence, if the battery dies or the BIOS gets reset and nothing is plugged in to IPMI, your machine is basically completely insecure to anyone on the same network.
They said this wasn't a security issue and they wouldn't fix this. Their reaction really turned me off to Supermicro. I ended up buying loopback plugs and installed them in every server I administer to avoid this.
So what do I do now? I run serial consoles on my servers and connect a small SBC, like a Nano Pi or Raspberry Pi that's only configured with ssh keys on IPv6. Since most UEFI implementations support serial consoles, you can do almost as much as we could do with real Unix servers of the past.
Happens a lot more than you’d think.. and for more than just servers (switch gear, routers, appliances etc..)