Here is a simple test: imagine you can press either button A or button B. If you press A 10000 people are spared the severest migraines. If you press B one person is spared a mild scratch. Which should you morally do and why? My answer is: you should do A and the reason why should do A is because we can compare the outcomes in terms of wellbeing and outcome A is comparatively much better than outcome B. What is your answer and why? Anyone denying that interpersonal wellbeing comparisons are possible or that we can in any way aggregate wellbeing ("there is no such thing as general utility") seems forced to say that there is no way to discern a moral difference between choice A and B. Is that really what you believe? If yes how do you put that into practice with regard to choices in everyday life?
As for backfilling, it seems to me that some people express aversion to utilitarianism, and portray it as very different from what it really is, as a way to shy away from taking moral responsibility for the consequences of what they do and omit to do. In fact utilitarianism is in the world as it is making similar demands as several other ethical theories https://twitter.com/ben_j_todd/status/1463867302089302019