I always turn off swap and that solves this problem. I really don't understand why it is on by default anymore. If you need swap, you are doing something very wrong somewhere else.

So, it depends on how much RAM do you have. Also with a swap enabled system can swap out some very rarely used memory pages, and cache some frequently used files instead - so by disabling swap you rob yourself of this opportunity :)

I have 64 GB RAM on my workstation, yet i still have swap enabled (but with lowered swappiness value).