They have many similarities -- they both redrew the current computer architectures, they integrate memory and computing, they can have randomness built-in, and they both take less power than mainstream GPUs.
What does memristor provide that specially designed "neural FPGA" can not?
Presumably, a memristor based neural network would have the advantage over an FPGA of requiring significantly less silicon area to achieve the same function. I imagine an FPGA based neural network would approximate analog signals digitally, perhaps using floating point "half's" or something. Memristors would directly operate on analog signals, encoding information as amplitudes or pulses of currents and voltages.
Notably, FPGAs and GPUs can't really be directly compared to each other in terms of power consumption unless you specify specific use cases. You can't build the equivalent of a mainstream GPU out of FPGAs without severely limiting the clock speed (because of how physically large it would be), and if you did anyway, it would use many orders of magnitude more power to function. So, a GPU is way more power efficient than an FPGA for rendering graphics. There are certainly problems that a GPU isn't good at solving, and so there's a good chance that an FPGA solution would be more power efficient.
I believe there is a great value in being able to "snapshot" the state and later load exactly the same state into millions of devices. And I cannot see how this will easily work with memristors.