I don't have direct experience, but I work in video production and have researched building an SDI capture device that would use an FPGA on a PCI card.
There's a lot here: - VHDL for the FPGA - DSP algorithm - A driver for the OS you are building for - Scopes and measurement devices for the hardware bits
If you tried to do it yourself you'd spend more money and have a bad time. I'd recommend getting an FPGA dev board and start there.