We're doing this with H.264 at work. Pretty useful since we need to generate the encoded stream anyway, so now we get motion estimation for free.
We were originally looking at analyzing the h264 bitstream in software and getting the vectors out that way. Honestly I don't think that would be extraordinarily difficult, there's already code in ffmpeg to draw the motion vectors as arrows on the decoded frame, so that would be a good place to look if there aren't any preexisting apis or tools to extract them.
Good luck!
https://github.com/NVIDIA/video-sdk-samples/blob/master/Samp...