What's annoying and counter to popular convention / user expectations is that you have to keep your finger in the established path to keep grip of the "ball". (S'pecially given your finger often blocks the view of it).
The feedback is immediate and highly discoverable.
The Lock Screen on your phone has totally different criteria (ease, muscle memory) than a toy that uses the same metaphor (fun/challenging)
Also: isn’t it more fun this way??
Meanwhile, I find typing anything on my phone very annoying.