Yeah, the current solution is similar to overfitting, this wont generalize to harder math where the operation doesn't correspond to the activation function of the network.
I don’t know, but cannot jump to your conclusion without much more domain knowledge.
It was just a simple feed-forward network. It can't do arbitrary amounts of repeated addition (nor repeat any other operation arbitrarily often).
Otherwise, you'd need up to 255 (or so) additions to multiply two 8 bit numbers, I think?
[1] https://en.wikipedia.org/wiki/Multiplication_algorithm#Examp...