You'd be surprised at how many mathematicians are avid hikers. I've heard stories of people flying to Calgary, disappearing in the Rockies, then showing up at UBC for a conference over a month later.
I have a friend who did a math post-doc at Calgary. When he "went to the lab" to get research done, "the lab" was a mountaintop.