Just asked my office mate who's been using VIM on ubuntu for probably 10 years (vim, maybe not ubuntu) now about this. He literally edits and builds code all day every day for 8+ hours. His environment is pretty simple, not many plugins, a solarized them and that's about it. He mostly develops in css, html, javascript, lots of python, and c++. Never had a segfault. This matches my experience as well.

As others have said, how in the world are you having these kinds of issues?