Does this mean that you could dehumidify a room very quickly by just vent-ducting a blower fan to your dehumidifier? Or is there a point of diminishing returns where the air must linger for a certain time within the dehumidifier to actually have any humidity extracted?

(Yes, I'm aware that such a set-up would be impractically noisy; it's just a thought experiment.)

A dehumidifier already has its heat pump capacity minimized for its design airflow.

An air conditioner has its heat pump capacity sized to cool a house, far in excess of what is needed merely to remove the humidity.