This hits the sweet spot. It's more flexible than a DSL like Caffe which gets in the way as soon as you stray even a little bit, without requiring as much boilerplate as a generic framework like Theano.

I have a question:

When training LSTM, how do you handle the gradient calculation?

I believe the original approach is to do a single step of BPTT for all the weights outside the unit, and do RTRL for the weights inside each unit. Is that what you've implemented?