Oh yeah! Thanks for asking cause this was the dilemma that we were experiencing at first. When starting out, think we were using mediapipe from ML Kit (Google) and CoreML. If you want a visual example of our first version, here it is:
https://www.swivi.app/.
We started out on Android but switched to iOS because everyone in our area had iPhones lol. In short, the skeleton (the white lines) kept flickering a lot so we made our own implementation. Feel free to check out our iOS version history too!
You can see our tracking capabilities from this stupid short: https://www.youtube.com/shorts/cGJs9pJW_nE (Skip to the last part where I wave my arms around quickly)
You're very right, the general body pose estimators are laggy and doesn't track very well. We worked full-time to implement our own solution that took YEARS XD. So this isn't smth that we just made. Admittedly, we were focused on building which I know isn't the best approach. But..at least, we got this tho ahaha
For our full iterative building process, you can view it here: https://www.youtube.com/watch?v=j5CAK40wTwo&t=17s
But the most important aspect that you pointed out which I liked is safety. Like all AI, you shouldn't completely trust it 100%. The app is good at what it does, but it's just a tool that can assist your needs.
I tried to keep it brief, so I hope this is answered your points. Appreciate you!