Their system works well on a RISC-V, which is a deeply pipelined but in-order architecture running at only 50MHz, with very fast memory relative to its clock speed. I'd like to see results compared to a more reasonable CPU implementation. For example they could have used a Xilinx Zynq, which includes hard Cortex A9 cores, which are out-of-order and superscalar. They also run at a much higher frequency relative to the memory speed.

I think this paper vastly underestimates memory constraints of higher performance systems.