Yeah that was always how it was sold to me too. But then the Dark Times came.
It still works a bit that way, but the parts you can really see are all of the metric dysfunction. And I blame Schwaber for setting this avalanche off. Of all of the Agile processes, Scrum seems to have the most surface area for attack by metric dysfunction.
I didn't think I'd end up missing Jeff de Luca so much. He banished story size estimates a long time ago, making Scrum feel like a step back to me. I've only recently heard of people rediscovering that you only have to split up stories that are 'too big' and you can still do fairly accurate estimates and projections based on the aggregate behavior of 100's of stories.
What I see missing or unstated in virtually every discussion is that time isn't the constrained resource. It's energy. There are lots of tasks I could complete in half an hour, but the strain of doing so would still mean I'd only get 2 things done that day. There are also things I can do in 4 hours that still leave me spent for the day. And plenty of managers who will try to make you question your self worth for not volunteering to do 2 of those things in one day because 4+4=8.
Think about that the next time you catch yourself on Hacker News and you scold yourself for wasting time. I'd lay even odds you're either recuperating from something you just finished, or frustrated by something (or being blocked by something) and taking a break.
Think about it when you're trying to articulate why you accepted 60 hours of work this week because they're trivial stories with a lot of wait time built in. They're trivial because they're intellectually and emotionally 'cheap' tasks. There are boring tasks that take every fiber of your attention and you will refuse to double up on those.
Because it's energy, not hours. Always has been.
And you know what else takes a ton of energy and social capital? Arguing with some fucknut about whether a story is 3 points or 4.