If you take "only suffering matters" to its logical extreme, you get Antinatalism, which essentially says our goal as intelligent beings should be to stop existing in order to minimize suffering.

Yes, suffering matters, but certainly it shouldn't be the sole basis for morality.

Suffering exists so that you don't get burned twice. Remove suffering, die in a fire.

This is sort of my thought process as well; the universe is unjust as long as suffering exists in any form at all. Whether this has any sort of inherent “meaning” is sort of beside the point, because suffering doesn’t have to have deeper meaning to be horrible in and of itself.