They don't need to: if you keep going straight in an infinite city you'll eventually reach a place that's virtually indistinguishable from the one you intended.
And just wait until you reach the library!
And even if you go to a specific place in an infinite city, the entire trip will require at most one turn, an infinitesimally rare event. No wonder we never catch them in the act.
If you watch for long enough they also sometimes fail to stop. So just like a real city too.