fastest to slowest: zsv (0.07), xsv (0.16), goawk (0.42), python (~1.6)
Obviously, does not tell the whole story as this test was limited to "count" and an interpreted language is expected to always be slower compared to a precompiled command, but, it might be relevant to a user deciding what tool to use. Also, might be instructive as to room for improvement in the go code (or possibly the go code could use the c lib)-- I note that even if the goawk command is '{}' the runtime is still about the same.
full results:
---
goawk:
1000001 7000007 real 0m0.435s user 0m0.435s sys 0m0.031s
1000001 7000007 real 0m0.413s user 0m0.419s sys 0m0.024s
1000001 7000007 real 0m0.425s user 0m0.430s sys 0m0.024s
xsv:
1000000 real 0m0.157s user 0m0.141s sys 0m0.013s
1000000 real 0m0.156s user 0m0.141s sys 0m0.012s
1000000 real 0m0.158s user 0m0.142s sys 0m0.013s
zsv:
1000000 real 0m0.066s user 0m0.053s sys 0m0.010s
1000000 real 0m0.077s user 0m0.060s sys 0m0.012s
1000000 real 0m0.069s user 0m0.056s sys 0m0.010s
python:
1000001 7000007 real 0m1.589s user 0m1.553s sys 0m0.026s
1000001 7000007 real 0m1.583s user 0m1.550s sys 0m0.025s
1000001 7000007 real 0m2.122s user 0m1.675s sys 0m0.037s
---
The script for this was:
---
echo 'goawk:'
(time goawk -i csv '{ w+=NF } END { print NR, w }' < worldcitiespop_mil.csv) 2>&1 | xargs
(time goawk -i csv '{ w+=NF } END { print NR, w }' < worldcitiespop_mil.csv) 2>&1 | xargs
(time goawk -i csv '{ w+=NF } END { print NR, w }' < worldcitiespop_mil.csv) 2>&1 | xargs
echo 'xsv:'
(time xsv count < worldcitiespop_mil.csv) 2>&1 | xargs
(time xsv count < worldcitiespop_mil.csv) 2>&1 | xargs
(time xsv count < worldcitiespop_mil.csv) 2>&1 | xargs
echo 'zsv:'
(time zsv count < worldcitiespop_mil.csv) 2>&1 | xargs
(time zsv count < worldcitiespop_mil.csv) 2>&1 | xargs
(time zsv count < worldcitiespop_mil.csv) 2>&1 | xargs
echo 'python:'
(time python3 count.py < worldcitiespop_mil.csv) 2>&1 | xargs
(time python3 count.py < worldcitiespop_mil.csv) 2>&1 | xargs
(time python3 count.py < worldcitiespop_mil.csv) 2>&1 | xargs