You could run this analysis on a laptop very easily.
The pile-10k dataset they used for analysis is 33MB, and GPT2 runs ok on a CPU. For the full 10K analysis it's probably quicker to get a GPU though.
The pile-10k dataset they used for analysis is 33MB, and GPT2 runs ok on a CPU. For the full 10K analysis it's probably quicker to get a GPU though.
This is correct. We did almost all this work on a Macbook Pro. Although for the pile-10k dataset analysis we used an A100 GPU because it would take many hours to run the whole thing through GPT-2 on a laptop.