Show HN: Skymind Intelligence Layer Community Edition

Hey folks - Skymind co-founder here...

If you download SKIL CE, you get a deep learning environment that supports neural nets from notebooks to production. The Zeppelin notebooks let you configure, train and evaluate neural nets and build ETL pipelines. It also gives you one-click deployment. SKIL comes with an AI model server that auto-elastically scales to handle surges in data traffic. The model server is architected as a micro-service and exposed through a REST API and takes input data as JSON and outputs decisions about that data as JSON.

One of the hurdles data science faces is that its hard to integrate data science tools with the enterprise big data stack on the JVM. We built that bridge. And we integrate with JVM big data tools like Hadoop, Spark, Kafka, ElasticSearch and Cassandra.

The main features in SKIL CE that aren't in the OSS are:

* model version history (easy cloning)

* grouping and indexing neural net experiments and bookmarking champions and challenger models

* scalable AI model server

* managed Conda environment

SKIL imports models from Keras and other DL libs, and it will have a managed Conda environment for Tensorflow, Keras and other tools, as well as the libs we built around Deeplearning4j.

ICYMI, we contributed DL4J etc. to the Eclipse foundation last month. Those libs, bundled in SKIL CE, include:

* Deeplearning4j: Neural network DSL (facilitates building neural networks integrated with data pipelines and Spark)

* ND4J: N-dimensional arrays for Java, a tensor library: "Eclipse January with C code and wider scope". The goal is to provide tensor operations and optimized support for various hardware platforms

* DataVec: An ETL library that vectorizes and "tensorizes" data. Extract transform load with support for connecting to various data sources and outputting n-dimensional arrays via a series of data transformations

* libnd4j: Pure C++ library for tensor operations, which works closely with the open-source library JavaCPP (JavaCPP was created and is maintained by a Skymind engineer, but it is not part of this project).

* RL4J: Reinforcement learning on the JVM, integrated with Deeplearning4j. Includes Deep-Q learning used in AlphaGo and A3C.

* Jumpy: A Python interface to the ND4J library integrating with Numpy

* Arbiter: Automatic tuning of neural networks via hyperparameter search. Hyperparameter optimization using grid search, random search and Bayesian methods.

You could say that SKIL is RHEL for AI. It's commercially backed open-core software. It's a deep-learning tool aggregator that addresses a lot of the current pain points of data science, and includes model auditing and tracking for enterprise compliance.

Right now we can import models from Keras 1.x and 2.0, no matter which backend you trained on, including Tensorflow, Theano and Caffe. In our next release of SKIL CE (3 weeks-ish away) we're going to enable people to train and deploy with TF directly.

All available here: https://github.com/deeplearning4j

The community is active here: https://gitter.im/deeplearning4j/deeplearning4j