you can scale vertically with a much better tech than awk.

enter duckdb with columnar vectorized execution and full SQL support. :-)

disclaimer: i work with the author at motherduck and we make a data warehouse powered by duckdb