It lacks intent and understanding so it can't be racist. It might make racist sounding noises though.
A fine example ... https://www.youtube.com/watch?v=2hUS73VbyOE
They still lack intent/understanding. But a system does not need understanding for its output to consist of racially biased statements.
But let's also point out that GPT can also make very powerful anti racist statements. It can persuade people to be less biased against race. Reacting to the specific beliefs and points raised in the conversation. Generating sensible counter arguments.
In that way, GPT4 is better at refuting racist rhetoric than most people are.
So it's potential here is both good - its ability to help the fight against racism - and bad - its risk of generating racism.
Shoehorning "anti racist" messaging into the tool makes it a substantially inferior tool.
That makes it a more useful and powerful tool.
How do you know? Well that requires verification, which is probably more costly than doing the legwork yourself in the first place.
That applies to all LLM generated output. It is worthless until the reasoning is validated. Which is costly.
It can simulate a decent project or product manager, a requirements analyst, a test planner, a junior implementer, debugger, automation engineer, etc
If you can't make your engineering pipeline nicer with those kinds of roles, I think you would either struggle to direct a team of those same people IRL, or you're just not being candid and effective enough with your prompting. Read papers on promoting ideas like reflection, critique, tool use, etc. Really treat it like a valuable team member, and it will boost your product requirements, write great tests right out the gate, and many other things you will WISH you could get your human colleagues to do! And all for a tiny fraction of the cost of human specialists!
For example, if a <skin colour 1> person things that all people of <different skin colour> are basically the same but all seem to be more intelligent than people of <colour 1>, it's definitely a racial bias but is it really racist to think that a different group of people have an advantage somehow?
Arguably it's still racism, even though it's your own genetics you're putting down rather than other people's, but as an example: if a black person in the USA said "I don't think I'll try to go to university, it seems white people find academic work easier" I'd call it internalised racism, or racially biased, but I wouldn't call that person "a racist" even though I disagree with them. Then again, if they started going round trying to convince everyone else that black people aren't as clever as white people, then I would consider them racist despite being the skin colour they're being racist against. To me it's about negativity towards a group vs. misguided thinking, rather than about whether it's against people like you or not.