This would work as a sort of energy storage - you could overheat your water reservoir during cheap power times and then transfer the extra heat to the entire city the rest of the day.
You would need a lake sized insulated reservoir for that to work. Too much water is used already by those systems, they don't typically want to add another storage. Typically they are gigawatt scale systems burning something because just using electricity is too much strain on grid, unless you have a power plant nearby (like cogeneration, which uses waste heat from electricity generation from fossil fuels).
Why? Just overheat the water circulating through the district water system when you can. People's thermostats will simply turn off sooner, leaving more heat in the water.
Then later, you underheat the water, and the city's various thermostats turn on a bit longer.
The risk is that you underheat too much, and people compensate by plugging in their own heaters during underheating. And underheating will occur during expensive energy periods, so this will exacerbate periodic energy shortages. But overall this could work to store a ton of energy - there's a huge volume of water in a district heating system's pipes.
Some places already used flooded mines or other underground reserviors for this purpose. As an added bonus, they're already warm and continually have heat replenished by geothermal heat.
