No they don't. Their - or anyone's - ability to restructure society is (or at least ought to be) constrained by the requirement to respect the rights of those in that society.
I agree that it ought to be, but to a lot of people, those rights end when they involve behavior outside unspoken social norms. The concept of rights and freedoms has always brushed up against that limit. There's been constant change in what those norms are, but changing them has never been easy.
