This is one argument I never understand. Why would you care if you can carry thousands of books at once? I rarely read more than one at a time.
I don't use ebooks. Physical books are still more attractive. You can abuse them without worrying about damaging them. And most importantly, when you have finished reading a spectacular book, you can give it to a friend to read.
Because you don't always know what you'll want to read next, especially while traveling.
Additionally, most eBook users don't just read books on their devices, but also newspapers, magazines, blogs, and articles from services like Instapaper and Pocket. (And for what it's worth, you can lend most eBooks.)
Like others, I do often find myself flipping between books on a trip depending upon my mood and level of energy: serious lit, brain candy, non-fiction, etc. It's nice to have the choice on a device (tablet) that I would tend to carry with me in any case. I also like having an illuminated screen. The lighting is often terrible in hotels and other locations.
That said--if I didn't travel a lot--I'd probably be relatively ambivalent about e-books. I like the de-cluttering aspect but I'm probably not going to re-read most books anyway and I find e-books generally a suboptimal format for things like cookbooks that I do keep around.
Second, even if you only read one book at a time (like yourself, and me as well), an ereader like the Nook or Kindle is still significantly more compact than even a trade paperback. A 700+ page hardcover? Forget it, it's not even a contest.
But 2-3 is a far more typical use case, and even at that scale, the carry weight matters. Plus people are carrying phones (and increasingly, tablets) around these days, in any case. So that's where the value proposition kicks in.