I disagree with the first part. The only thing less useful than estimating points is estimating hours.
No task significant enough to warrant a ticket in a queue and design effort takes less then a day. Even if you you swear it's "done", the more likely outcome is you'll be dealing with it for hours later that week when some sort of an issue crops up.
I've seen so many people "bid" 0.5 and 0.25 day units of time (in points or whatever) and then act offended when I challenge them on that, yet 3-4 days later they're still plugging away at the same task due to "complications".
And any task that is meaty enough to be a task will take at least a day, and probably longer.
The one place where I will more or less disagree is when bug-fixing. There are always some bugs where you pop open the debugger and have it solved in an hour or two. I've been at places where bug report tickets were handled differently from stories/tasks, though, so maybe this is fine to just think about separately.
Asked in another way - why is it more useful to estimate in an unit that's more abstract/distant-from-reality?