Think of a very very elliptical orbit, one that's nearly a line and consider what happens when you take a bit of energy off the orbiting object. At perigee, when the energy is almost entirely kinetic, you're mostly just changing the velocity, rather than altitude of the object. At apogee, when the energy is almost entirely potential, it's the other way round - the change is almost entirely one of altitude. This is further stretched by the much more 'visible' effect of potential energy changes on altitude than of kinetic energy changes on velocity.

the atmospheric drag at perigee happens perpendicular to the object speed, which is tangential to the gravity field, so the object doesn't get shifts on it's vertical speed, only it's energy, having less to convert in potential energy (i.e. apogee height)

in the thereabout of perigee you have two arms, one inbound and one outbound. the horizontal component of the drag for both is all in the direction against the travel and slow the craft without changing the vertical component of its speed

the vertical component of the drag (for as long as the craft has the energy to get's outside of the dense atmosphere) kind of cancels out, as the drag force inbound pushes against your down velocity vector and the drag force outbound pushes against your up velocity vector.

there is however a small energy loss due to the fact that the journey trough the atmosphere isn't perfectly simmetrical, since you're losing speed along all the way, the inbound leg is shorter than the outbound leg, causing the vertical component of the drag going outside to slightly exceed the component of the vertical drag coming in - so your perigee does change little by litte due this (and other more nuanced effects, but we're painting with a large brush here) - this asymmetry gets stronger the more you stay in the atmosphere and the furthest you dip in the dense area, until you ultimately don't have the energy to escape anymore.

the fact that the atmosphere isn't on/off like that but has a long boundary layer stretching far from the dense parts kinda complicates things a little more, because drag is applied the whole circle, but not by that much.

The rule of thumb you can use to make this intuitive:

A change in velocity at a certain point x on an orbit will change the entire orbit, except that orbit will still continue to go through that one point x.

So slowing down at perigee will not alter the perigee. (It is actually the most delta-v efficient way to lower the apogee.)

That is about it.