Typically when I serialize data for #index endpoints I return much slimmer payloads than I do in #show endpoints. I recognize that 120mb was an extreme, but you should be able to serialize lists of 500+ items into well under 1mb, with enough data to render your list view thumbnails/links.
I always thought the infinite scroll pattern was designed for analytics purposes. So that you can track user engagement. Not because technology constraints necessitated it or that it was a good user experience.