Great, thanks for the pointers! I've tried momentum trick before and it has helped. I'll try rmsprop.