While I agree that's a hassle, I don't think this should result in a refund. It's like ordering dinner, eating it, and then wanting a refund for the meal because you waited a long time for the meal.
It's like if you were paying for a meal by each minute taken to eat it, then the waiter left your food on the counter for an hour before bringing it over because they were busy, then they charged you for that hour because "it already left the kitchen"
Adding insult to injury caused by their own lack of preparedness.
I’m not paying just for food. I’m paying for the entire dining experience. I also probably have plans after the restaurant that are now delayed or a babysitter waiting at home for me to return. If you not only don’t deliver on a good dining experience but also inconvenience me I should get a refund. In my experience, restaurants generally understand that and I don’t even have to ask for the refund.