The other day my boss asked me to find out if there was any second where we processed more than 100 transactions a second on our server in the last 90 days. I needed to go through 100s of GB of data which mawk managed in no time and in a relatively simple one liner. I imagine my idiomatic python script would still be running
Often times, analysis is an evolving thing. As you uncover one answer, new questions arise. It’s far easier imo to tweak my command line pipelines than to modify program flow.
At the end of the day, general purpose programming languages are jacks of all trades and masters of none. They’re inherently held back by their flexibility when compared to DSLs. Text processing is so easy in awk b/c it’s a text processing domain specific language. If you really think that we should use on language for everything, you’re ignoring the value that specialization can provide you, whether in speed, power, or ease of use