You're absolutely right, the thing to look at here is the St Petersburg problem.

You have a coin, and each time you don't get heads the payout doubles. Get heads and get the payout. What do you pay to play this?

It turns out that you can't work out an E[X] for this, as it's a series that doesn't converge. But also if you go around and ask people what they'd pay, surprisingly they don't say infinity. Expectation is thus not the last word in decision making.

Turns out utility is a thing that matters, and we can come up with some thoughts on it (von Neumann-Morgenstern etc).