How would you query the location where you need to load more data when scrolling down (the highest empty spot)?
The other posters have good answers. One thing to consider for a smooth interaction would be to eagerly load the next x elements before they scroll into view.