I won't say I'm religious about it--especially if somewhere familiar--but if I'm being doctrinaire, I'd argue you should always have a physical map and compass while hiking.
With respect to compass, I'm not sure how critical real orienteering is for most people. It's more about not being totally turned around and/or being able to make sure you hit some feature like a road.
The little handheld trail GPS my parents got me after my first such misadventure has proven incredibly useful for figuring out where I made the wrong move. (The first such misadventure, I was using the paper map to work out if there was a bend I missed).
As for orienteering: I learned it once and never used it. (btw, the Sierra Club course where I learned it got cancelled when someone sued over an injury in ice ax practice.)