People are perfectly free to use Apple App Store and stay free from viruses and have a nice curated environment. Nothing changes for them if you allow sideloading. The ONLY change is that you have the OPTION to run software Apple doesn't officially sanction. That's just a basic freedom right there and it's hard to see any rational argument against it.
The rational argument is that the ability to side-load applications would be exploited by the nefarious and the clever to take advantage to naive users.
If a website can convince someone to download a "new version of the Flash Player", those same websites will convince someone to "side-load this application to protect against identity theft!"
Android allows sideloading and this doesn't happen (too much) in Android. It's possible to support walled garden by default while giving the user advanced options to sideload, which are difficult but not too difficult to find.
For one thing, that is how you get viruses. This is amusingly similar to the mask refusal arguments.
> People would get bad software on their phones, but last I checked, this is happening already
"My proposal would lead to bad things, but there already are bad things" is a terrible argument.