The better way to understand libertarianism is to characterize regulation as something like "a rule anybody makes up and then punishes people for violating it even if they never agreed to follow it." Now you don't need to resort to weird contortions to handle cartels and things because they're just a form of government and therefore something to be limited in order to restore the benefits of free market competition.