Danny Hillis discusses the thinking behind locating it in a remote mountain here https://soundcloud.com/longnow/progress-on-the-10000-year-cl...
But the entire talk is worth listening to.
It's not intended to be a secret - it's there to be visited (I hope one day to visit it myself), the mountain is part of a pilgrimage to better frame the experience.