The "vision fps" runs around 12 to 20, so yeah, that's gonna seem kinda jumpy. The display fps is slightly higher, which contributes to a juddering feeling.
I think the more interesting/dangerous jumping is when the perception engine fails to consistently classify/find objects in the scene, between adjacent frames, not the specific fps of the process. This video seems to display some of such inter-frame jumpiness.
Inter-frame jumpy-ness can be smoothed out slightly by the control algorithms, on the assumption of object permanence, as long as the underlying vision NN doesn’t miss things too often.