I've had a lot of luck at American Eagle, which is the only store I've ever found adult-sized clothing that was too small for me (their shirts sometimes go down to XXS, their pants and shorts to 26", though not every item comes that small).