It depends on the treatment method your city uses. Chlorine gas will evaporate out, but chloramine won't, and will also react with a lot of things downstream to make nasty tasting organics, for example if you use the water in brewing.
And can depend on where in a city you are. And sometimes the different treatment systems are still mained to eachother so shifting demand and supply can change which water you’re drinking from.