I stand corrected! I shouldn't have guessed "quadratic" without measuring.
Large files will make gcc use more memory, and if you're building on a low-RAW machine, the OS will start to swap. That's probably the effect I saw. Nothing to do with "quadratic" compilation time.