As an end user, who cares? Make it more like Keras. Quit messing about w tensor optimization because you'll get smoked by someone else's performance eventually.
Last time I checked, there wasn't even a function to shuffle a tensor. Something you'd do at the start of every training iteration.