The reason it needs all permissions is that some people have their own pages and the data is on there. So you might need permission to crawl those secondary pages for twitter and github (?)

At least I think that's how it works. I could be wrong.