People are responding that I'm using the word "we" and therefore I'm not being objective. I think I need to emphasize a point better: there are infinitely many objective measures. Here's one: how many total paperclips humanity produces over the lifespan of the universe. That's a truly objective measure; we can just count them. We can analyze every action any human ever takes and estimate whether it increases or decreases the total number of paperclips any human will ever create. There's no subjectivity at all -- it's completely objective.

While objective, paperclip count is notorious for not really aligning with our intuition of good vs. bad; I'm pretty sure we can do better. But this illustrates the point that the problem is really about picking from the infinitely many objective measures that, objectively, do already exist.

Humans exist; that is objectively true. Humans have preferences; that is objectively true. Humans take pleasure from some experiences and experience pain from others; that is objectively true, albeit a big simplification. It's objectively true that humans dislike having rusty nails jammed into their eyes. It's objectively true that humans enjoy eating nice meals when they're hungry. We should seek to perform those actions which minimize the number of rusty nails jammed into eyes and maximize the number of times humans can enjoy nice meals when they're hungry. Obviously there are trillions more considerations, but nowhere has subjectivity even entered the picture yet.

What I'm saying is that we can pick, or define, or refine, a truly objective measure; one that is much better than paperclip count. You could say "well there's subjectivity in picking which measure we use " -- only because we have imperfect knowledge of the universe and its future. If we had perfect knowledge of all the likes and dislikes of all humans that will ever exist, all sources of pain and pleasure and suffering and contentment and sadness and happiness, all their values and goals and desires, and how all actions affect all of that, then we could simply integrate their effects over time. Yes, we have to weight them appropriately, and find a good NPV discount rate, and all that, but with perfect knowledge, we could easily find one that's at the very least on the Pareto surface. That's our metric.

So our subjectivity comes from imperfect knowledge. But that's OK -- we accept that we have imperfect knowledge. There's a huge difference between "there's no objective measure of good vs. evil" and "it's really hard to pick an optimal measure of good vs. evil, and to know which actions maximize the net present value of that metric over the course of the universe." The former gives you nothing; the latter at least orients your moral compass toward something that humanity can all work toward together.