I don’t think anyone is objecting to what you did as much as how you did it, and how you seem to be proud of flagrantly abusing your ability to duplicate other people‘s intellectual property. I’m hardly a champion of copyright laws or IP in general, but running duplicates of someone ese’s site feels completely wrong to me without thinking twice. Like the suggestion from the pen tester here, which you posted on your blog, this would be a lot different if you had written the article about conduct that seemed professional, respectful and legal.
Great question. How about we invert that, and you tell me what IP laws justify operating a functioning duplicate of someone else’s entire website, full of copyrighted and trademarked content, for the benefit of your business?
By this logic, I could duplicate any website in the word and operate a copy for my private business. While I am not a lawyer it seems clear that this is not legal (and as if this is the first time the concept occurred to someone!)
I assume archive.org falls under Fair Use. Check these guidelines.
Duplicating your competitors website for analysis to benefit your business fails the first condition. If it were academic research or some sort of public benefit, that’s different than for-profit republishing for your SEO business.
