It's understandable, but in no way nice. One side is going to bring their authentic shy and antisocial self, and stonewall the invitations, while the other side needs to keep smiling and send invitations no matter what. This sounds slightly lopsided, doesn't it?

If you would like the other side to do you a small favor every time, it's worth considering to do the same. At least respond to the invitation with gratitude and a hope to maybe do it next time.

You can overcome shyness to some extend. Not getting invited anymore can also be a sign that the shy person has to change something about their behavior, instead of all others just accepting that.