The code is very easy to read. Doesn't seem like there's data/model parallelism support for training, which will be important for real-world use.