The FPGA manufacturer has characterized all their package pin delays. It's possible to export a csv file with internal delays of the package for each pin from the FPGA design tool. With Xilinx Vivado it's just File -> Export I/O Ports.
Edit: it's using the pad property of "pad-to-die-length" if doing it manually, right?
I did some prototyping but never got the stage if actually fumbling around with PCBs a few years ago, things seemed to progressing quite well.