Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The difference is that when you search on Google - at least before AI overviews - you end up at the source site.

Also Google respects robots.txt. Every site that Google surfaces chose to be in the index.





Thats not entirely true. Google might or might not hide your pages from index. They'll definitely going to scrape it anyway. They also display summarized info from your page (famous "what is scrapping" joke showing wikipedias summary). Finally, you might just get your answer without visiting - just by skimming result description.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: