That doesn't matter. Eventually the dust will radiate too.

Yeah, you are right. It could be in frequencies we cannot observe with the naked eye, but eventually, we would have to tweak some other postulates, too. Let's start over and postulate a gigantic turtle that carries the world…

> Alternatively, most of the stars really far away could be covered in a Dyson sphere.

The principle of energy conservation dictates that all the energy inside the sphere gets out, in one form or another. So the existence of Dyson spheres, dust clouds, dark matter etc. only delays the appearance of the energy in the universe at large.