AGPLv3 refers to "all users interacting with it remotely through a computer network", not "people who are interacting with someone else's running copy". If someone adapted AGPLv3 into a new license that requires all "people who are interacting with someone else's running copy" to be able to access a copy of the source code regardless of whether the interaction takes place over a computer network, that new license would still be free and open source. The FSF has not bothered to write such a license yet, but anyone can do it.
This part of the FSF's page on free software endorses this type of rule:
> Rules that “if you make your version available in this way, you must make it available in that way also” can be acceptable too, on the same condition. An example of such an acceptable rule is one saying that if you have distributed a modified version and a previous developer asks for a copy of it, you must send one. (Note that such a rule still leaves you the choice of whether to distribute your version at all.) Rules that require release of source code to the users for versions that you put into public use are also acceptable.
This part of the FSF's page on free software endorses this type of rule:
> Rules that “if you make your version available in this way, you must make it available in that way also” can be acceptable too, on the same condition. An example of such an acceptable rule is one saying that if you have distributed a modified version and a previous developer asks for a copy of it, you must send one. (Note that such a rule still leaves you the choice of whether to distribute your version at all.) Rules that require release of source code to the users for versions that you put into public use are also acceptable.
https://www.gnu.org/philosophy/free-sw.en.html#packaging