It is worrying when a private company has access to this sort of information.
It is urgently alarming when a government has access to that same information.
Governments have a long, proven history of abusing that information in ways that result in undermining of democracy, immoral imprisonment of political opponents, mass murder and genocide, and other niceties.
Governments should be afraid of the people, not the other way around.
Imagine a full list of sites you visit being stored by your ISP and now available to your government when necessary. Wow.
Does this mean users will be flagged automatically if they visit sites that offer pirated software/movies and the like?