Which is insanely far away and all light coming from there will have diminished in energy enough to be barely visible?
Inadequate explanation. Even if the universe was a closed box with perfectly reflecting borders, what you would actually see in the sky is something like an average brightness of the entire universe with some fluctuations.
It's not inadequate. Brightness diminishes for two reasons. One is absorption from cosmic dust (not that big of a factor), the other is that the object appears smaller, but the brightness per unit solid angle from the object does not diminish. If there were just more stars behind that you could see, the sky would have to be much brighter.
Photons do not "diminish" in energy except via red shifting (where does the energy go? into the expansion of the universe!).
Light scatters, gets absorbed and re-emitted, it's true, but if distances don't change then the universe would still be much brighter than it is today.
Any straight line you draw from earth to any direction in the sky will then reach a star.
Only if you think that there is infinite matter, all of it forming the same kind of stars, uniformly spread over the infinite space and there is no unknown form of atenuation for waves traveling millions or billions of years light.
