It's fair to say we will have to replace this with something not developed yet. It's honest and if you are scared by it others will come up with that solution.
I' ll offer a guess at a solution.. the next generation of power sources will be organically based and breakdown into non-toxic material and solar will be a part of that.
Solar panels are even better: they don't break down into anything, ever. They are completely inert, pre-vitrified from the factory and guaranteed to hold their shape for millions of years.
And when they're old and lost most of their efficiency, one can still use them in less demanding applications. Example: take old panels from households or industrial buildings and use them on parking lots for services or lighting. To a certain extent, the same applies to batteries too.
"We might be replacing one problem with another. We need to move on from toxic solar to some unknown fuel of the future."
Notice the classic FUD sowing, ambiguity, and of course an oblique reference to some as-yet-developed economic synthetic fuel.