That's not what I think he's saying. I think he's saying that the whole point of Beamr is to not have to choose a fixed 2 Mbps bitrate. Thus the test should be:
So the post should contain:
1) a random selection of high quality original videos (30-40 Mbit).
2) those videos compressed with Beamr.
3) those videos compressed with x264 - ALL WITH THE SAME OPTIONS - those options tuned to match the average bitrate of #2.
Notice the difference between #3 and what Diaz has done: he has tuned the parameters manually for each and every video file. IMHO that's a rather harsh comparison: "Hey scumbag! Your software is worthless because a really smart and dedicated human can do it just as well/better!!!"