This statement is a bit broad and thus debatable in overall accuracy, but it is based on something real. It is probably overly centered on the western and white-centric world, since being white or not doesn't even come into question in many other parts of the world. But in the US/Europe...

Women have been, for valid reasons and patriarchal overprotectiveness, been trained to feel vulnerable and exposed when walking outside. Always half-focused on who may be looking at them or tailing them, it takes away the mental freedom needed for perambulatory reflection.

Black men, brown men, immigrants, similarly, have over time faced a variety of hate crimes and false incarceration, that they must again often pay attention to their perceived "suspiciousness" factor, such as wandering around late at night with a hoodie on, or idly staring into a park where kids may be playing, lest they be mistaken for a latent criminal and get apprehended or worse.

So "walking has been the privilege of white men" is a bit of an extreme, but "justifiably walking thoughts free with impunity" is indeed the privilege of white men in the lands of white men.