Show HN: Juggernaut – Experimental Neural Network in Rust | Hacker News Reader

Show HN: Juggernaut – Experimental Neural Network in Rust

juggernaut.rs

54 pointsafshinmeh19 comments

I'm a bit confused. How useful is this if:

- Rust cannot compile to the GPU

- Neural network programs are usually not large and therefore do not need the type safety that Rust offers

- All cool neural network research is done on Keras/Tensorflow, so developing on that platform gives access to new algorithms automatically

- Scripting in Python is virtually at least as fast as anything else because you can use Tensorflow which uses the GPU

> All cool neural network research is done on Keras/Tensorflow

Add in PyTorch if you are really talking about nn research.

> - Neural network programs are usually not large and therefore do not need the type safety that Rust offers

TensorFlow Fold [1] uses static type checking at model compilation time, presumably because it is meant to consume structured data, and needs to know what the opaque tensors it passes around are supposed to represent.

I don't fully comprehend that type system, and debugging type errors was annoying (huge stacktrace full of passing type info around), but after I managed to imitate it well enough to hack up my own blocks, I didn't investigate further. A language-level type system like Rust's would probably have made things easier for me.

EDIT: I just remembered that there's also Leaf [2], which claims to be among the faster neural network libraries (although their comparison is a bit outdated). Evidently there are people seriously pursuing machine learning in Rust (still uses CUDA for GPU acceleration, though).

[1] https://github.com/tensorflow/fold

[2] https://github.com/autumnai/leaf

Author of Leaf here. Yes, Leaf has been dead for more then a year. There recently has been an effort by some community members to pick up development of Leaf under a new name: https://github.com/spearow

> Rust cannot compile to the GPU

Rust might not be able to compile to the GPU today, but any language can target the GPU, it is only a matter of tooling support.

> All cool neural network research is done on Keras/Tensorflow, so developing on that platform gives access to new algorithms automatically

I disagree with that assessment. The implementations that come with research are rarely of a quality that they can just be picked up and used. Yes, the community is bigger with Python frameworks, which gives you quicker access to the new stuff, but the effort is about the same as writing a new implementation in another language.

It doesn't matter. Most of the time, especially with Tensorflow, you will have a static computation graph somewhere. Nothing (well, implementation-defined) prevent you compile that into CoreML / Caffe / Caffe2 or just Tensorflow lean version to get deployed. PyTorch is a bit different because it supports dynamic graph.

which gives you quicker access to the new stuff, but the effort is about the same as writing a new implementation in another language

Not really. You can just serialize a Tensorflow graph, freeze the variables as constants, and then load and run the graph using Tensorflow's small and convenient C API. There are bindings to the C API for multiple languages (e.g. Rust and Go).

This is how I use neural networks in my Rust (and formerly Go) programs: I just build and train the graph in Python and then use it in Rust.

Newer versions of Tensorflow also have the XLA compiler, which compiles a graph to executable code that you can link directly into an application. I haven't tried this approach yet, since the C API serves me well, but it looks promising.

#1 is not strictly true, although it's not necessarily useful yet: https://github.com/japaric/nvptx#nvptx

I'm confused by several of the API choices in the example. Why is the training set part of the network? I would have expected it to be a parameter to the train() function. Same for the activation function, shouldn't this be a property of the layer rather than fixed for the network as a whole?

I get that this is just in the early stages and more for learning than anything else, but it doesn't seem very well thought-out IMO.

Thanks for your suggestions, yorwba. I do see what you mean about the API and I will work on this.

For now, if you want to train a model, you need to pass data to the NN struct not train method but as you mentioned, probably it is better to pass it to the train method.

Sorry for the confusion and thanks for your comment!

Keep going! I really like being able to follow projects that start small, as opposed to 'here is my 10000 line toy project.

That being said, you will probably get some flak, probably because of the insane amounts of rust evangelism people on hn have had to deal with

What is your goal in writing this project?

There's nothing to see here.

Trivial NN implementations are a dime a dozen, and this one is no different. It's just a partial work in progress; it's not a 'Show HN'; it's just a few hundred lines of toy code.

...and that's the same feedback it got on /r/rust last week.

I don't see why it's turned up here now.

(Just as a baseline, at this point, if you can't use your NN implementation to at least do a basic classifier on MNIST, its probably not worth showing people)

Basically because it's ticked off all the buzzword bingo that works on HN. Unfortunately that's how it goes to front page.

No not really no! I'm trying to gather some information and feedbacks about my project. I don't care (at this stage of the project) to be on the first page of HN or not, as long as I have some good feedbacks to improve it. Thanks anyway!

thanks for your suggestions, shadowmint. I tried to mention it is "experimental", not a production ready NN.

I'm working on adding more features to this project but for sure, it is not production ready at all and is not a competitor of Tensorflow or Leaf.

Calling this an experimental neutral network is like me calling an opengl demo that renders a few flat shaded polygons an 'experimental 3d engine'.

It's not.

I'm sorry if that sounds harsh, but that's what you have.

There's no point to showing off something this raw: all you'll get is negative feedback about the limited features it has and negative feedback from people who try to use it for something and find that you can't actually do anything with it.

Just hold onto it until it can actually do something useful with it; and as I said in my previous post, I'm not talking some 'abstract' sort of useful thing.

The MNIST dataset is the basic standard task for these frameworks; my advice is focus 100% on being able to classify that dataset, and then you'll have something to show.

Yeah, you are right. Maybe it's even less than something to be called "experimental".

I will devote more time to this and I will try to make you happier with the demo page and further versions.

Reply on news.ycombinator.com