Like staunch mentions, there are cheaper places to live in than either SiV or Boston. Living in Austin-TX would cost half as much for instance. Living in developing countries would cost even less (at least if one doesn't insist on 100% equivalent quality of life).
Disclaimer: we're based in the valley and are happy for it, but that's because we're profitable anyway.
I did this. I built the prototype for my first startup in Goa, India, using GPRS internet (unlimited access for a fixed price, because it was a beta).
I even recommend it, assuming you can connect with real alpha customers before moving over there, or if you have a bizdev guy stateside. I had to download overnight but it was workable. I spent my days floating in the pool reading books like 'Design Patterns' and Java/Eclipse books to help me understand Eclipse well enough to build my app using it. I programmed by night. I got the prototype done.
The ticket over from Atlanta was $1200 or so. My rent was $400 a month in a 4 star resort on the beach (off season), and that was my only appreciable expense because food was basically free. There was even another entrepreneur with a PhD there doing the same thing, and he had brought his entire book library with him in an ocean container!
BTW - I came home with about six feet (when stacked) of books, because they were so cheap. A good technical book in paperback is about $5 at the TATA bookstore at IIT in Bangalore. I couldn't help myself, I bought a complete library and two suitcases to carry it home in :)
Disclaimer: I heave a 'weak stomach' and I got dysentery twice, methanol poisoning once and other weird stuff in between, but mostly they are avoidable if you get your shots, go vegetarian (they make it EASY), and be super strict about bottled water.
I met several hackers who moved to India to be able to work on whatever they wanted without doing much in the way of consulting. One guy was there productizing Israeli mesh networks to supply wifi and VOIP to Tibetan refugees in Dharamsala. Another guy was building FOSS tools for college professors. There is an entire subculture of people that do this.
Just watch out: its easy to get distracted. And check your calendar and remember the monsoon is no joke :)
Disclaimer: we're based in the valley and are happy for it, but that's because we're profitable anyway.