If the current is 3 mph, and you can swim at 4 mph, then you will be swimming for a very long time at the effective 1 mph to get back to shore. And if on the way you need to stop for even a minute's rest, you'll lose 4 minutes worth of your hard-won progress.