This is why native apps use recycler views for not just infinite scroll, but anything that can display more rows/columns/items/etc than can fit on screen at once. Recycler views only create just enough cells to fill the screen even if you have tens of thousands of items to represent, and when you scroll they reuse these cells to display the currently relevant segment of data. When used correctly by developers, these are very lightweight and allow 60FPS scrolling of very large lists even on very weak devices.
These are possible to implement in JavaScript in browsers, but implementation quality varies a lot and many web devs just never bother. This is why I think HTML should gain a native recycler widget of its own, because the engineers working on Blink, Gecko, and WebKit are in much better positions to write high quality optimized implementations, plus even if web devs don’t use it directly, many frameworks will.