FPGAs are hard/expensive to program (including being very painful to debug), at least for most tasks - and few people are good at both FPGA and software development, so you may need different people to do the programming. New FPGA generations can require substantial re-work, at least for best performance. Many tasks would require kernel support (which is another "hard/expensive to program, and annoying to debug", although not nearly as bad).
You need a deployment scenario where FPGAs beat GPUs (and CPUs) by enough to be worth that pain, but not by enough that someone already has a ready-made ASIC for you. I agree that one would expect to see more deployment of FPGA's naively, but not a lot.