I don't understand the jump from: classical algorithm takes time A -> quantum algorithm takes time B -> (A - B) must be borrowed from a parallel universe.
Maybe A wasn't the most efficient algorithm for this universe to begin with?
Right, and that's part of the argument against quantum computing being a proof (or disproof) of the many-worlds interpretation. Sure, "(A-B) was borrowed from parallel universes" is a possible explanation for why quantum computing can be so fast, but it's by far not the only possible explanation.
Maybe A wasn't the most efficient algorithm for this universe to begin with?