Which works fine for iPhone owners, and on the 15% of Android devices running Marshmallow, in the relatively rare case where someone knows this can be done and acts upon that knowledge. Everybody else is hosed.
You can disable the location access entirely on Android, and enable temporarily only for the short moments when you really need it.
Most Androids have a slide-from-top quick menu when you can toggle it with one click. Honestly, Google Maps is the only app on my phone that really needs location access.
Just because you turn it off doesnt mean it's not tracking you. Have a look at google location history, it doesn't use GPS but it can still pinpoint you very closely using cell tower triangulation. Stallman was right, we carry the worlds most advanced tracking device in our pockets.