Modern USB requires you obey the 150 mA limit before enumeration. You are supposed to be switching power to the rest of the device that needs to exceed that limit.