Hacker News Reader

Comment by IshKebab | Hacker News Reader

How difficult is it to set up custom recurrent networks? The only system I've ever seen that handles this well is CNTK - you can just say `x = PreviousValue(y)`.

I've tried to work out how to do similar things in Torch and Tensorflow but all they really offer is pre-packaged layers like LSTM. If you want to make your own it's difficult, undocumented and not at all ergonomic.

How does Caffe2 compare?

View on HN

I am a complete machine learning noob, so this could just be my lack of skill. Having looked at most of the popular ML frameworks, none of them seem to provide an easy, functional way to implement a custom 'neuron'. That is:

  function (someinput):
      output = do_stuff
      return output

Which then gets passed on to the next layer. Again, it's quite possible I've just missed something, or there is some inherent ML limitation that makes this impossible. But if such a thing is possible, it would be a rather awesome feature from my POV.

Many do. For example, the `Lambda` Layer from Keras:

https://keras.io/layers/core/#lambda

Yeah tried that one a while ago but didn't have much success. I'm a bit of a dummy when it comes to ML. That said, it looks like they've updated the documentation or something, so might be time to give it another whirl. Thanks for the reply.

tensorflow's py_func allows you to do that, you can also write your own C++ operators, their API interface for that is pretty nice. In fact, you should be able to pretty much write any op you need by combining existing ops (maybe not as efficient as you'd like but yields the results you want).

Sorry I'm late to reply. This looks perfect for what I have in mind! Thanks!

This is another area CNTK's 'BrainScript' excels. You can pretty much express it as normal maths:

https://github.com/Microsoft/CNTK/wiki/BrainScript-Activatio...

List of functions:

https://github.com/Microsoft/CNTK/wiki/BrainScript-Full-Func...

The recurrent network work is under way - we are revisiting the designs to consciously balance between performance and API niceness. We'll share more tech details in the upcoming days. Right now, this is an example RNN:

https://github.com/caffe2/caffe2/blob/master/caffe2/python/e...

We use it for machine translation so the perf is nice.

If you have design feedbacks please let us know - we'll be really grateful.

I know one of the biggest struggles of training RNNs in TensorFlow, for instance, is the fact that the graph is static. Other frameworks, like PyTorch, have dynamic graphs.

Will Caffe2 support dynamic graphs (perhaps down the line)? What is your reasoning behind whichever decision?

Reply on news.ycombinator.com