Okay, and as soon as I switch to JSON, I have not just 5 million referrers logged per day, I also have 5 million times the word "referrer" in my log. Nice.
Trivially solved by compression. In those five million referrers you’re probably also repeating “Firefox” and similar strings over and over, so compressing logs is already standard practice.
Compressing logs that are being written to? Never heard of that...