The end of human suffering also represents the end of the (IMO vastly greater) aggregated joy of being alive as a human.
Why even give birth to humans that are most likely not wanted in this world?
I think most people can achieve happiness in their situation, even under bad conditions, if they have a sound mind. Those elites might have a better shot at accidental happiness, but being upset over that will do nothing good for your own state of mind, so why let it bring you down? You can't change how the world works.
Happiness comes from within, and from meaningful social connections.