Private libraries are the privilege of those who do not move frequently, or at all. A large library is surprisingly heavy and bulky to move, and each time you move you will be tempted to leave something behind. If the odds of leaving it behind are even 5% then over time the odds approach 100%. That said, the physical copy of a book has its own story, the dog-eared pages and cracked spines, the occasional stain. I personally somehow find myself remembering which side of a book something was on, and roughly how far through. Very handy since it reduces the search space immediately by 2, at least. And of course, you get all the benefits of messy, imprecise search - the joy of finding something new. Plus you can easily lend them or give them away, and they can't be remotely deleted.

The OP writes as if ebooks and paper books (and audio books!) live in perfect zero-sum disharmony. It seems like most inventions have this effect on discourse. This is more a comment on the https://en.wikipedia.org/wiki/Gartner_hype_cycle than anything. In this case, I think large paper libraries will tend to dwindle, but not disappear. Much like how accurate music recording and playback caused live music to dwindle, but not disappear, etc.