The extremely detailed mapping of drinking water on OSM that the OP alludes to, is more helpful than you assume. If you are cycling or hiking in baking-hot Southern Europe, or wandering through some picturesque villages, there are often springs hidden just off the trail that most people would never spot, if the map didn’t alert people to their presence. Also, with OSM one doesn’t need to be “chronically online” and use a “web service”, because various apps allow using OSM fully offline.