I was admitted to university through an EOP (Equal Opportunity Program), which is in the same vain as DEI. One of the criticisms of the program was just that-- people felt like they didn't deserve to be where they are. A constant worry that people would view you as the "one that got in because of their skin/income/background, etc). Obviously this applies to DEI as well.
As someone who benefited from EOP, it's weird for me to say this but in all honesty I think we would all be better off without it or perhaps doing it in a way where it isn't labelled or "celebrated" the way it is... it should be unobtrusive and not call attention to itself.
I was one of three white people out of 350+ people that was accepted through EOP. If I felt that way as a white person I can't imagine how much more amplified those feelings would be as a person of color.