I like your standard but I think the real criticism being made is not that it's arbitrary but that it's arbitrary in different ways for different languages. This starts to look like intentional dishonesty. Another standard closer to the status quo would be "anything portable". Why does one language require portability and not another? What is that actual problem with the example given? There doesn't appear to be one.
> Why does one language require portability and not another?
Because more than one language implementation was measured for that language but only one language implementation was measured for the other languages.
If only one language implementation was shown for Ruby it would be Ruby 1.9 - not JRuby
If only one language implementation was shown for Lua it would be Lua 5.1.4 - not LuaJIT
If only one language implementation was shown for Python it would be Python 3 - not PyPy
PyPy and LuaJIT are being treated more favourably than other language implementations.
