Exactly the same case here. A few years ago, pre-ChatGPT, my wife was working with a massive database in SPSS or R (can't remember which). Now these "databases" are really just huge CSV/TSV files, with millions of records, one record per line. She was applying some very complicated mathematical functions to every single row, and she was frustrated because it was going to take days to apply a particular calculation she was working on to the whole database.

So I took a look for about 30 mins figuring out the code (I don't know SPSS or R), and it turns out she was doing some calculations that were identical for every single line, regardless of that line's fields. So I pre-calculated a few things, prior to starting the per-record-loop, and then just used the pre-calculated values every time. After that, every run took less than 5 minutes. She's amazing at math and economics, but optimizing low-level code is not something she has done in the past.