Hacker News new | past | comments | ask | show | jobs | submit login

Downloading a DB dump and crawling locally is possible, but had two gnarly show stoppers for me using wget: the forum's posts often link to other posts, and those links are absolute. Getting wget to crawl those links through localhost is hardly easy (local reverse proxy with content rewriting?). Second, the forum and its server were really unmaintained. I didn't want to spend a lot of time replicating it locally and just archive it as-is while it is still barely running



Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: