Nice example. I have personally seen Hadoop jobs which would only scale by using larger VM instance types.
There are more solutions other than M/R now though (spark, presto etc) but they require high performance servers (presto suggested RAM amount is 128GB).