Sadly there is a particular type of person who discovers the day that nobody is supposed to go to meetings is the day when everybody is free and the meeting rooms are available. And then starts scheduling meetings for that day.
After too many days of back-to-back meetings from 10am to 3pm I've taken to creating a repeating lunchtime meeting with myself. Then I discovered the people who don't care if you're available or not...they consider themselves important enough that you'll drop everything for their meeting.
In this case, I'd say there's no/negligible cost to missing the meetings on a no-meeting day, unless the entire group of people have decided that it is worth breaking the no-meeting rule. Of course, you should let people know that you won't be there in advance...