Apart from the mass distribution, if you look at a car travelling on an uneven road from the reference point of the car itself, you will see the wheels moving up and down very quickly to keep contact with the ground. My understanding is that adding mass to the wheels will slow them down and thus increase the time they don't touch the ground, reducing the grip.
F=ma, so if you increase the mass you either decrease the acceleration (they slow down) or increase the force (they hit the bumps harder). In practice it's both. The vertical acceleration when you hit a bump is essentially determined just by your speed and the size of the bump; it can't be slowed down. So instead what happens is the bump will exert a greater force on the tire (which will deform it more, reducing grip and probably meaning you'll need to run higher pressure, which also tends to reduce grip). On the other side, when you leave the bump, the force is determined by the spring rate of the suspension and the amount of compression, so for the same suspension, a heavier wheel will move back to the ground slower, which yes, will reduce grip. You could increase the spring rate to compensate, but that would exacerbate the first problem, and also wouldn't help in the situation where you hit a dip from steady state rather than exiting a bump.