Let's try imagenet training. Intel's best time is 3h25m on 128 nodes. Imagenet is ~150 gb.
(150 GB)/(3 hr * 3600 sec/hr * 128 nodes) = less than one megabyte per second per node! Caffe is slow! CPUs are slow!
Or bs metrics are bad?
You can see the workflow for yourself and see how the task is quite parallelizable.
ETL can be very extensive. For example, we first built this when we had to take data from 15 different database systems that represented individuals and their pension contributions and join across these systems. It was a largish join about ~4 tables from each system so around a 60 table join.
That was "JUST ETL". The job was preparing the data for training. ETL is often times a large part of people's workflow. Looking for needle situations in a haystack. That can be JUST ETL. If you believe there is a more apt word for extracting data from a system, performing unspeakable transformations on it, then making that information available to another process then please tell me. Being of Peruvian stock myself I take great license with my language and grammar.