The first option is how "game AI" has been developed by most gaming companies until recently. The second option is how DeepMind, OpenAI, and a few others are now building gaming AI. Only the latter is about building a gaming AI that "learns to play the game like a human does".

I have seen a few cases where they build a client for the game.

It receives messages like a connecting client application does and sends messages to the server as well.

The downside of this is that you are trying to honor an unpublished API that no one promises not to change and no one will tell you when it does.

IIRC it's almost entirely based on game APIs or memory hacking

The latter is much harder. It has been done for e.g. Atari games.