The Reduceron specifically tries to quickly perform application of lambda expressions that GHC will try to avoid generating in the first place. The Reduceron speeds things up using several memory banks etc. but it still does things that shouldn't be done at all and the overhead is there at least in area and power.
There are situations where a FPGA is better suited of course (very low latency real time signal processing for instance) but for general purpose computing FPGAs are not exactly ready for primetime IMO.
Adding to what simias said, even an FPGA with built-in floating-point primitives can beat a GPU (in floating-point-heavy computations when the measure is performance/cost) only if the algorithm doesn't fit well onto the GPU architecture – for example, if you can make use of the highly flexible SRAM banks on the FPGA. I suppose there exist such workloads, but they're rare.
Also, keep in mind that no FPGA comes even close to the raw external memory bandwidth of modern GPUs.