It's been a long time. But I remember I started with this page [0]. There are also many resources on github, like [1] or [2], but I haven't tried them.

0: https://trac.ffmpeg.org/wiki/Debug/MacroblocksAndMotionVecto...

1: https://github.com/LukasBommes/mv-extractor

2: https://github.com/jishnujayakumar/MV-Tractus

There is a github project called pikrellcam that utilizes motion vectors generated from the raspberry pi's gpu to detect motion. It's a relatively simple piece of code and it works very well