I've thought about this before - where you have a 2D array of images with infinite scroll in all directions.

You need to track the scroll position, and kill the elements that are scrolling away as you create the new elements in the direction you are scrolling - with the entire purpose of creating enough cached images so that it is seamless to the user.

The problem is that the scroll position changes back when you kill the old elements, so you need to keep a running tally of the matrix somewhere to track the user's position.