You don't pipe the data directly through an LLM with this tool - it writes Python code that manipulates the data.
So this will handle millions of rows of data quite happily. I've used it extensively with SQLite in the past - you can upload SQLite databases to it and have it run SQL against them. You can also ask it to save data to SQLite for you to download.