TL,DR; Although I'm not a fan of entire Hadoop system, some judgements about it is totally wrong.

1) Do one thing well - Hadoop is so stacked up because of this (which has pros and cons). Every single component of Hadoop is designed specific to a job (there are many overlapped components, though)

2) Hadoop's main idea is to be data-local, yet this Pachy thing not to be: "Data is POSTed to the container over HTTP and the results are stored back in the file system". Data-locality (almost) always win.

3) There is no scheduler like YARN specified, yet fleet and etcd are not schedulers.

4) You may not like Java or JVM-based things (like I do), but there is no problem of being Java-based or JVM-based.

…