Hacker News new | past | comments | ask | show | jobs | submit login

but their robots.txt says that you're welcome to scrape it:

  User-agent: *
  Disallow: /ac.php
  Disallow: /ae.php
  Disallow: /album.php
  Disallow: /ap.php
  Disallow: /feeds/
  Disallow: /o.php
  Disallow: /p.php
  Disallow: /photo_comments.php
  Disallow: /photo_search.php
  Disallow: /photos.php



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: