Looks like this is a classification problem as opposed to a regression problem. (The NN is trying to pick one output). You very likely want to use a cross entropy loss function, not MSE.