I agree with your premise, but I am curious what apps are you side loading. Are these important apps? I value this freedom as well but I have only needed to install blokada. I am a working professional and do not tinker with android as much as I used too.
If you install apps from F-Droid (an alt app store on Android), they are all open source and built by F-Droid themselves. The benefit of this is, zero malware. And much better privacy. It's a nice peace of mind. The Play Store and App Store don't have such a good track record. And there's a lot of great apps on F-Droid (like clients for Twitter, Reddit, Hacker News, etc)
