American here: I've only heard "desi" used as a self-descriptor by South Asian people, usually Indian (but I think Pakistani sometimes as well?). It doesn't carry a negative connotation here that I know of, though I might have missed one that does exist.