Totally agree on OpenCV's Python API being hard to use. If your goals are to build something as foundational as OpenCV, but with a Python-native interface I'd be excited about that.
I hope I don't come off as critical, I appreciate the work you're doing. I'd really like to see this take off. My only point is that tasks like annotating a video with tracking are things I've only seen in demos. If I could custom-order the reusable parts I want, it would include geometry, camera transforms, lens distortion, etc. Your polygon zone filtering looks imminently useful. Maybe I should shut up and just contribute something.
I remember meeting you! Maybe I'll see you in Seattle this year.