Ask HN: What's the most frustrating part about saving money?

A few months ago I asked the HN community what their favorite way to save money is. People really seemed to engage with the topic, so I'm revisiting saving money from a different angle this time.

I read last week that something like 50% of American households don't have enough cash readily available for a $500 emergency. This seemed crazy to me, but it definitely seems like there's a real problem relating to saving money these days.

So - what's the most frustrating part about saving money??