Looks like Daiz did a similar test to the one I described below. Naturally we would need blindly randomized videos and randomized audiences, rather than just Daiz audience of one knowing exactly which videos were encoded by what. We'd also need some function converting the CRF parameter to the Beamr Perceptual Model parameter ("minimum possible" parameter), which we could obtain by knowing the Beamr parameter for a given CRF when Beamr and x264 choose the same exact bitrate.

randomized blind A/B testing really can make all the difference. they did the same on the Hydrogen forums, for testing lossy audio codecs, and if there was only a hint of information which bit was which, results would be heavily biased.

crazy, but that's how human perception seems to work. it also raises the question whether, at some point, this placebo effect might not be way stronger, than any actual perceptual differences left over after correcting for biases.

and if that's the case, maybe we don't need better codecs, but shinier TVs, and better people to convince us that video quality is better and more enjoyable. like snake-oil salesmen. or like that high end electronics brand that put a weight in their remote controls, just so it feels more 'solid' and high quality (brilliant idea, I forgot what brand, might've been Scandinavian).