Think of it this way. Is browsing to a random HTTP address via IP on the internet and then screencapping the picture produced on your browser legal?

Then using a VNC client in the same way would fall under the same legal purview. I think as long as there is no interaction to carry out functions, attempt password/username combos, then it's fairgame.

A prosecutor armed with the CFAA would probably disagree with you.

Browsing to an address with your browser is like checking to see whether a shop on main street is open right now. Attempting to connect to a an address with VNC is more akin to walking around the back of a house and checking if the rear door is locked or not.