In my mind it came from the French non plus, meaning neither. Often heard when responding "me neither", as the "me" would be a very short syllable at the beginning of the response. I can imagine people hearing "non plus" and seeing uninterested or unfazed people uttering it, sharing lack of enthusiasm with someone else.