Semi-related question: If I write a spider for Tor and my spider collects CP (NOT by design but just in the course of spidering) how responsible am I for:
* Having that data in my DB
* Showing said site in search results
I assume the answer is something like: "You're not as long as when you do notice it you report it and then delete it off your servers (Maybe also blacklist the Tor URL so it doesn't get re-spidered/indexed).
I haven't written anything to do this but the concept is extremely interesting to me but I'd hate to write something, let it run, forget about it (keeping it running or just holding on to the data if I ever want to do something with it), and then getting in trouble down the line for having CP or other illegal material. I've tried googling for this but couldn't find anything good.
TL;DR: Are search engines (On the web or Tor) responsible for the content of the sites they index
* Having that data in my DB
* Showing said site in search results
I assume the answer is something like: "You're not as long as when you do notice it you report it and then delete it off your servers (Maybe also blacklist the Tor URL so it doesn't get re-spidered/indexed).
I haven't written anything to do this but the concept is extremely interesting to me but I'd hate to write something, let it run, forget about it (keeping it running or just holding on to the data if I ever want to do something with it), and then getting in trouble down the line for having CP or other illegal material. I've tried googling for this but couldn't find anything good.
TL;DR: Are search engines (On the web or Tor) responsible for the content of the sites they index