Rephrasing beambot, some of the algorithms employed use knowledge of the next frame (or two or five) to filter the current frame. So for those it doesn't matter whether you use a FPGA, DSP, ASIC, commodity processor, whatever to do the filtering - you have to wait until you have the needed future frames to even start.