That's the interesting part to me. I have very very very slight glasses. If I never put them on, it's fine. If I've been wearing them, and THEN take them off, everything looks horrible.
I mostly only wear reading glasses at work.
Happens to me when I switch between contacts and glasses despite the same prescription (well, not that everything looks horrible, but that I notice how accustomed I am to the peculiar way glasses shape the world for a few hours). I wouldn't attribute too much to it.
You'll even notice it if you change the type of lens, to a higher or lower index.