It's incredibly satisfying to reproduce these papers. I now make Rust versions of the most interesting projects. And try to make low-latency inference pipelines for those that show potential for real-time use. Some are sketched out here: https://github.com/Simbotic/SimboticTorch

The bulk of the work to get real-time working is to move more of pipeline to GPU. Mostly things handled by numpy and some image/video transformations.