Having built something similar with CLIP on an M1, frame sampling rate is the whole ballgame. One frame a second on 12k videos is days, keyframes only got me to an overnight run.
Based on my experience, sampling rate can be tricky if what you are looking for lasted less than interval period.
you are pirating first-release movies for commercial purposes?