Could you please explain why? It's an additive update to the weights, adding an outer product of a vector with its transpose, which must have rank 1. What am I missing?
You can’t diagonalize a generic matrix by adding a rank 1 matrix. That holds only for dimension 2.