Very interesting experiments. I have been thinking for a long time that it's a good idea to do these atomic tests. About test 8, it is actually well known that if you want accuracy vs # of epochs, decrease the batch size -- however it causes time inefficiency. And I found number 7 quite surprising!