> The primary goal of Speedometer 3 is to reflect the real-world Web as much as possible, so that users benefit when a browser improves its score on the benchmark.
As with any other benchmark its results will be interpreted incorrectly and will have little effect on real world.
Google already has vast amounts of real-world data. The end result? "Oh, you should aim for a Largest Contentful Paint of 2.5 seconds or lower" (emphasis mine): https://blog.chromium.org/2020/05/the-science-behind-web-vit... Why? Because in real world the vast majority of sites is worse.
Browsers are already optimised beyond any reasonable expectation. Benchmarks like these focus on all the wrong things with little to no benefit to the actual performance of real-life web.
Make all benchmarks you want, but then Google's own Youtube will load 2.5 MB of CSS and 12 MB of Javascript to display a grid of images, and Google's own Lighthouse will scream at you for the hundreds of errors and warnings Youtube embed triggers.
Optimise all you want, and run any benchmarks you want for the "real world", but Lighthouse will warn you when you have over 800 DOM nodes, and will show an error for more than 1400 DOM nodes (which are laughably small numbers) for a reason: https://developer.chrome.com/docs/lighthouse/performance/dom...
1. These companies themselves don't practice what they preach. No matter how fast Speedometer 3 is, Google's own web.dev takes three seconds to load a list of articles, and breaks client-side navigation. Google's own Lighthouse screams at you for embedding Youtube and suggests third-party alternatives [1]
2. The DOM is a horrendously bad, no-good, insanely slow system for anything dynamic. And apps are dynamic.
There's only so much you can optimise in it, or hack around it, until you run into its limitations. The mere fact that a ToDo app with a measely 6000 nodes is called a complex app in these tests is telling.
And the authors of these tests don't even understand the problem. Here's Edge team: "the complexity of the DOM and CSS rules is an important driver of end-user perceived latency. Inefficient patterns, often encouraged by popular frameworks, have exacerbated the problem, creating new performance cliffs within modern web applications".
The popular frameworks go to extreme lengths to not touch the DOM more than it is necessary. The reason the DOM and CSS end up being complicated is precisely because apps are complex, and the DOM is ill-equipped to deal with that.
This only goes to further show that browser developers have very little understanding of actual web development. And this is on top of the existing problem that web developers have very little understanding of how fast modern machines are and how inefficient web tech is.
This brings us neatly to point number 3:
3. Much of the complexity on the web in the modern web apps is due to the fact that the web has next to no building blocks suitable for anything complex.
https://open-ui.org was started 3(4?) years ago by devs from Microsoft and you can see just from the sheer number of elements and controls just how lacking the web is.
So what do you do when you need a proper stylable control for you app? Oh, you "use inefficient patterns by modern frameworks" because there's literally no other way.
And even if all of those controls do end up being implemented in browsers, it will still not be enough because all the other things will still be unavailable: from DOM efficiency to ability to do proper animations to ability to override control rendering to...
As with any other benchmark its results will be interpreted incorrectly and will have little effect on real world.
Google already has vast amounts of real-world data. The end result? "Oh, you should aim for a Largest Contentful Paint of 2.5 seconds or lower" (emphasis mine): https://blog.chromium.org/2020/05/the-science-behind-web-vit... Why? Because in real world the vast majority of sites is worse.
Browsers are already optimised beyond any reasonable expectation. Benchmarks like these focus on all the wrong things with little to no benefit to the actual performance of real-life web.
Make all benchmarks you want, but then Google's own Youtube will load 2.5 MB of CSS and 12 MB of Javascript to display a grid of images, and Google's own Lighthouse will scream at you for the hundreds of errors and warnings Youtube embed triggers.
Edit:
Optimise all you want, and run any benchmarks you want for the "real world", but performance inequality gap will still be there: https://infrequently.org/2024/01/performance-inequality-gap-...
Optimise all you want, and run any benchmarks you want for the "real world", but Lighthouse will warn you when you have over 800 DOM nodes, and will show an error for more than 1400 DOM nodes (which are laughably small numbers) for a reason: https://developer.chrome.com/docs/lighthouse/performance/dom...