I think what Dror is saying that if you take #3 and run it through their tool you'll get a smaller size with the same quality. So it's some sort of h.264 post-processor... I agree evidence would be nice.
That's not what I think he's saying. I think he's saying that the whole point of Beamr is to not have to choose a fixed 2 Mbps bitrate. Thus the test should be:
So the post should contain:
1) a random selection of high quality original videos (30-40 Mbit).
2) those videos compressed with Beamr.
3) those videos compressed with x264 - ALL WITH THE SAME OPTIONS - those options tuned to match the average bitrate of #2.
Notice the difference between #3 and what Diaz has done: he has tuned the parameters manually for each and every video file. IMHO that's a rather harsh comparison: "Hey scumbag! Your software is worthless because a really smart and dedicated human can do it just as well/better!!!"
bjornsing, I think you might not understand what Diaz is doing. He is not manually tuning the parameters for each video file. He is actually letting x264 automatically choose the bitrate and encoding parameters, to achieve a desired level of quality.
The way I understood it is they say they optimize bit rate for a given perceptual quality and their secret sauce is that they measure perceptual quality in a way that approximates what people actually see more closely. They "aim" for the same quality in the video you feed them and try and optimize bitrate. An example would be finding coefficients in a macroblock that could be further quantized without sacrificing perceived quality. So to me it makes some sense though I can't put a number of what you could gain with this sort of secret sauce, that is how much is there to squeeze out of already well optimized videos that use other measures. I would imagine the applying this secret sauce to the original video would be a better idea since quality measure is inherently something that depends on a reference. Almost by definition any change to a compressed video is reducing its quality (but perhaps they "reduce" things you can't see).
Since this is about perception it's really hard to measure. I looked at some of the comparisons people have done of the demo images and if you look carefully in some background areas you will see noticeable difference in quality between two images that people claim to be equal.
As your attention is naturally drawn to the foreground most people wouldn't notice that.
