I worked as a front-end developer for an adnetwork for two years, so I take a little responsibility for your experience. I tried my best, but there just wasn't that much interest in getting it right.
Here's why I think we're all having a hard time with advertising in mobile. Basically, adnetworks are still learning how to do it right, the same way designers and developers are still learning how to do mobile websites right. The layout issue you mention comes from a conjunction of various reasons:
* Marketing thinks mobile == iOS, which sets limited expectations and for example means developers have a hard time getting a variety of devices to test on
* Adnetworks are likely targeting mobile with the same front-end infrastructure they use for desktop. Most of them still target IE6 so they're trying the same ideas
* Old iOS versions and many other webkit browsers have big issues with fixed positioning. First they didn't support it. When they did, they still had issues rendering while scrolling and firing the correct events
* Ads in mobile require a complex layout but the standard in the industry is that the website doesn't have to do anything but insert a script tag in the page. If there was teamwork between websites and ads, they could use better layout tools like flexbox
Rooted Android phones can install apps[0] that block most ads (I think it only blocks connections to the ads and doesn't do any fancy CSS rules).
And Firefox on Android has Adblock Plus browser extension which as far as I know blocks ads like it does on the desktop version. No root needed for this.
When I open an app or browser on my smartphone, I'm usually focusing on a single task, where ads are intrusive and take away precious real estate from an already tiny screen. It's an enormous relief to use an app or site that just does what you want it to do and nothing more.
Ironically, it's on my smartphone that I'm most likely to use an app that's completely focused on advertising. When I'm looking for a movie or nearby restaurant, I love being able to read reviews and eventually move on to a dedicated web page where I can get more info (movie times, menus, etc.). There are apps that do this very well.
There are plenty of ways to get consumers to happily view advertising and promotional material, but inserting unrelated/unwanted ads into apps and websites just creates a frustrating user experience, especially on mobile.
I'll provide a different option: it takes too much time to get it right! iOS for example provides so many default widget toolkits complete with gradient, sheen, and dropshadow which make it really easy to create a polished experience quickly. The web has no such concept (well, bootstrap is sorta close but not nearly as polished and not for mobile). Why does the web not have these polished frameworks? Simple - ROI. The incentive for Apple and Google to develop said frameworks is large: a pool of developers who will take advantage of the toolkits and write much better / way more apps. What's the incentive for the web? Basically nil. Anyone can rip off what you did (one of the downsides to the web being open and standardized!), and you gain none of the benefit of enhancing your platform.
It does have them. There are many toolkits that provide components for the web. Bootstrap is just one of them and one that has been getting a lot of publicity here for a while because it's new and has a nice design. But there are many out there that cover different aspects of mobile web development: YUI, Sencha Touch/ExtJS, Dojo, jQuery UI/Mobile, Enyo...
And pre-made toolkits still require you to tinker with them to get layouts right, make them fit your design and make ads work OK in them. That's true for both the web and native toolkits.
Frankly, if all the ad networks in existence fall over and die from their own crudulence, I'll not shed any tears whatsoever.
Any site that uses crap ads that affect my user experience will really have to be something special for me to continue to use them. I'm happy to leave these sites for other sites that don't do stupid things to users.
Here's why I think we're all having a hard time with advertising in mobile. Basically, adnetworks are still learning how to do it right, the same way designers and developers are still learning how to do mobile websites right. The layout issue you mention comes from a conjunction of various reasons:
* Marketing thinks mobile == iOS, which sets limited expectations and for example means developers have a hard time getting a variety of devices to test on
* Adnetworks are likely targeting mobile with the same front-end infrastructure they use for desktop. Most of them still target IE6 so they're trying the same ideas
* Old iOS versions and many other webkit browsers have big issues with fixed positioning. First they didn't support it. When they did, they still had issues rendering while scrolling and firing the correct events
* Ads in mobile require a complex layout but the standard in the industry is that the website doesn't have to do anything but insert a script tag in the page. If there was teamwork between websites and ads, they could use better layout tools like flexbox
* No adblock on mobile (yet?) :P
Hopefully it'll change.