It's not that the barycenter is moving, it's that the orbit of the sun is not even remotely circular. The barycenter only moves in response to forces outside the solar system (the galaxy).
On the other hand, if you look at the barycenter of, say, Earth + Sun, then that barycenter will move around because there are a bunch of other planets involved. But the barycenter of the solar system is more stable, due to the conservation of momentum and all that, and the distance from the solar system to the nearest objects that would influence it.
On the other hand, if you look at the barycenter of, say, Earth + Sun, then that barycenter will move around because there are a bunch of other planets involved. But the barycenter of the solar system is more stable, due to the conservation of momentum and all that, and the distance from the solar system to the nearest objects that would influence it.