Going on vacation doesn't mean doing nothing.
Like you I don't understand how/why people do this.
I think it's about doing something different. Travel, visit, explore, camp, hike, do sport, meet new people, share that with family/friends or not.
Holiday goals, simply to have somewhere to go. They don't matter if you don't do them.
As you get to that goal, you take time to look around, maybe duck into a place here, do an activity over there. Or maybe not, and simply let life roll on by as you stroll to your destination.
I have gone hot air ballooning, day hiking, snorkelling with turtles. Wander over, have a chat, book it in for the next day or two. It is then the goal for that day and might lead to something else.
For me having a list of "places to be, things to do" means I have to be switched on, getting there and doing that, and if I don't then I have failed.
Holidays is noFail time. Allow serendipity to take charge.
You raised the question and answered it in two sentences. This is exactly why some people are able to take a complete break and "do nothing" - their daily life is already filled to the brim with work, family, kids, etc, that when they get on vacation, what actually feels different is doing "nothing".