crockpotveggies

DL4J contributor here. I spoke with our team so the differences in performance are likely explained by array ordering. ND4J, for good reason, requires F ordering because of limitations in cuBLAS. While I haven't had the opportunity to closely examine the Neanderthal comparison (I'm also not a clojure user), the likely explanation is that there's an implicit ordering impacting this.

Deeplearning4j is written around F ordering and ND4J supports this. Admittedly, our ordering API is not obvious to the average user.

Here's an example test you can run yourself that demonstrates ordering: https://gist.github.com/raver119/92b615704ca1bf169aa23a6a6e7...

  o.n.i.TensorFlowImportTest - Orders: CCC; Time: 11532 ns;
  o.n.i.TensorFlowImportTest - Orders: CCF; Time: 2101 ns;
  o.n.i.TensorFlowImportTest - Orders: CFC; Time: 10202 ns;
  o.n.i.TensorFlowImportTest - Orders: CFF; Time: 1960 ns;
  o.n.i.TensorFlowImportTest - Orders: FCC; Time: 10744 ns;
  o.n.i.TensorFlowImportTest - Orders: FCF; Time: 1717 ns;
  o.n.i.TensorFlowImportTest - Orders: FFC; Time: 10097 ns;
  o.n.i.TensorFlowImportTest - Orders: FFF; Time: 1716 ns;

We also profiled the above test and confirmed that F -> C ordering adds significant overhead. I can share screenshots if anyone is interested.

Yes, so one of the core libraries within the DL4J project is datavec which is ETL-focused. One key problem that we discovered - and fixed - was that reading and transforming data for training could bottleneck a multi-GPU process. You spend a lot of $$$ on a deep learning computer, but making the library performant enough so that you could load data at the same rate the GPUs could consume it was challenging. This scales to about 4+ GPUs (depending on datatype) and we're building a datavec server so this can scale much larger. There are still good returns if you clean and transform your data and presave to disk, which helps with large machines such as a DGX. However, other bottlenecks still apply (which we are solving right now).

I hope that answers your question. I consider the process of extracting records, transforming them, and loading them for training to be "ETL". I understand ETL also applies to other data consumption.

*I should also note that if you wanted to use datavec for ETL and do not wish to train a deep learning model, it is quite useful for columnar data.

The amount of work we've done this year with Deeplearning4j on performance has been much higher than previous years. We brought DL4J up to par with community standards while maintaining the advantages of Java. I think what a lot of people don't realize is that a ton of effort has been made toward ETL and integration tooling.

It's very difficult to train on multiple GPUs while maintaining performance of ETL. ETL is a scary hidden bottleneck.

I'm very interested to see how Eclipse can continue to push development. I think the people who will especially benefit from this are devops/production teams operationalizing data science.