Make the color blue then if yellow isn’t neutral enough (like iMessage tapbacks, or Facebook thumbs up). Make the people more abstract.
I’d argue the direction should be towards generality not identity specificity.
I’m not dogmatic and obviously my point of view on this is a lost cause, but I generally think the universalist ideal is better and should be the goal.
I do think yellow was intentionally chosen to be universal and not to be read as “white”.