First, the reprogrammability of FPGAs means lots of unused gates and less density, or wasted space. with flash technology nowadays, it doesn't waste as much power, but the footprint is just so large it's not worth it.
Second, a lot of the things that make custom silicon fast can be found in GPUs, such as ALU, MAC, FFT, FIR, SIMD and other DSP slices. Sure, there is a whole additional layer of optimization that can be done with custom silicon, but the computational powerhouses already exist. It's mostly (not all) a matter of reprogramming the memory movements from block to block, delegating certain operations to the CPU and general optimization. Most new codec algorithms can probably done pretty well with GPUs on phones these days.
And unfortunately, cell phone companies aren't interested in keeping older HW relevant :( Other industries might, though. My friend said lots of military radar projects he worked on used FPGAs.