Thanks for the tips and for the more or less confirmation of my terrible technique. You're right that I pay no attention to Philips vs Pozidrive, will definitely need to start taking that into consideration. When following a guide, I do pay attention to exact bit sizes when they are mentioned , but for guide-less jobs I just go with "this looks like it fits". Will try out wiggling before committing.
I imagine it'll take a while to start to get a feel for it, wonder what's the best way to get a good amount of practice without damaging (useful) things.