For most of human history, people fought over the right for members of their group to put up a hut or yurt on a piece of land. And/Or some other, more powerful group would come and take whatever they wanted, including sex and slaves, from other, weaker groups of humans.
Now you have a group (or multiple groups) of people that will use violence to prevent people from doing the things that they feel are wrong. Probably they'll decide on what those things that are worth using violence to prevent are, and probably they'll write them down so that everyone is on the same page. If they don't do that, then they are using violence for arbitrary and undefined reasons, which most people don't want. I think I see where this is going...
What is the vision for this scenario that doesn't just turn into new groups taking power and enforcing their rules?
The most important difference is everyone could create such a group without main authority monopolizing violence. Some would try to recreate current authoritarian relations, while others would oppose them.
It's a nice thought, many small bands of people each doing their own thing. But I don't see why people think it would turn out differently than it has turned out before, which is some groups becoming large and powerful enough that they push out, destroy, or absorb other groups, until they become large and powerful enough that they're just called government.
> Yeah, so indeed it's important to organize self-defense groups that can help you not get hurt or enslaved.
Did you realize you just came up with the system you were opposing?
We can organize into some groups, let's call them countries, with an organized self-defense groups, we can call them police and military, to prevent our yurts from being stolen.