Is PyGame flaky these days? I haven't much played with it in quite some time; it used to be pretty solid, although as I recall it wasn't much more than a wrapper for SDL 1.2 at the time.
See, for example how kivy has moved away from using it as a backend https://github.com/kivy/kivy/issues/5590