A game-playing AI could trivially be restricted to the same inputs as a human: send button events, but don't write directly to memory.

If it actually figured out how to execute arbitrary code via button presses (humans have done this in Super Mario World), then we should patch the bug and restart the experiment.

Of course, an AI gaining control over the real world is a different matter, because in that case it may be too late to restart the experiment.