I once got a talk demoted from a normal talk to a lightning talk at some unrelated conference to make it work with the schedule.
That was indeed not nice. But also not "Cruelty". I just acted professionaly and accepted the change.
"Hey we think you have great ideas to share, and would really like you to share them."
On short notice: "One of our team members thinks your ideas aren't that important after all, we are going to remove you from the headline. Please keep it short kthxbye."
It shows bad planning, bad organization, lack of cohesion, and outright disrespect of your professional time. You may choose to accept that slap to your face (signaling that it's okay to do that to you), but there's nothing unprofessional about rejecting the attendance and rightfully criticizing the organizers for their rudeness.
I agree that it’s bad planning. And in particular, the reasons of “discomfort” are really terrible. However, it’s a very small hill to die on. Like, a mound.
Others acting professionally is the best outcome for those who do not do so.