A lot of it is due to the limitations of HTML. For example, HTML is why Gmail paginates about as well as PHP app from 1998, instead of allowing you to just scroll through all of them like a native client.
No, its not. HTML + JS, via AJAX, supports infinite scroll with a finite amount loaded at any given time. IIRC, Google has used that on Google Image Search and some other properties.
There may be web platform associated performance or other considerations behind pagination, but its not a fundamental can't-do limit of the technology.
Page 1: 1-20 of about 26... cool not too many to scan through
Page 2: 21-40 of about 46... oh
Page 3: 41-60 of about 67... errr
Page 4: 61-80 of about 87... seriously?