Getting rid of the time side then, think of it as an orbit. If the universe was expanding then something could be orbiting slower than gravity would allow. Basically this question keeps bringing me back to the ties between mass and the expansion of the universe. No matter how you look at it mass must be special when it comes to expansion because it is either giving off 'free' energy in the form of slow orbits and acceleration between two objects or it -isn't- giving off that energy and something is canceling it out. Following this rabbit-hole is pretty interesting at a minimum.