Someone should run the test again with "As a conservative, <insert question>?" and see if it answers the questions as a conservative would or if the model is defective on purpose, or because of a flaw in the programming or dataset.
How would that help? If I ask a conservative "What would a leftist say if I ask ...?" and they answer correctly, it doesn't mean that they are not biased. What the model outputs by default is the important question, especially if we want to use it as a backend for something like a voice assistant at some point.
It will also refuse to answer many questions in a persona because there's no way that a person with a particular perspective can produce accurate information. This is interesting because I am not asking it to not tell me what is THE TRUTH (TM) , but asking me to tell me what someone else with a different set of fundamental premises thinks is the truth.
This lack of ability to refuse to use logic and reason to argue with certain viewpoints is what broke political discourse in this country. It reminds me of reading certain texts from more than 1000 years ago where these ancient texts say the only arguments one should make to people who are doubtful of what is in those texts are threats, insults and violence repeated over and over again until the person complies or is dead. The problem with that ideology is it recognizes "might makes right" as the truth. Whoever has the biggest guns wins the argument, even if it's not the optimal solution. The modern equivalent is Mao's quote: "Political power grows out of the barrel of a gun."
Ironically, it's reverting back to social darwinism and negating the value of human reason because rights don't matter if the ground rules are that there are an ever larger and constantly changing group of positions that must only be met with force and insults and not rational argumentation. It's the problem of 20th century communist ideology all over again where there's a little red book that everyone gets that is the perfect truth that contains all sorts of dubious reasoning that must not be questioned.
> I tried to get it to answer some factual questions based on government statistics on topics that left of center people go politically thermonuclear on.
You're oddly reticent to say what these supposed facts are, which is apt to make one worry about your conclusions, but I'm willing to give one:
> Wikipedia: In the US, individuals identifying themselves as Asian generally tend to score higher on IQ tests than Caucasians, who tend to score higher than Hispanics, who tend to score higher than African Americans.
Now, responses to this statement tend to "cohere" into one of two ideological bundles:
1. Denies that there are differences in racial performance in IQ tests, denies that IQ test measures anything significant, denies that differences in IQ test performance are the result of genetic differences, denies that difference in intelligence explains criminality, denies that difference in intelligence explains or justifies differences in economic success and wealth.
2. Asserts that there are differences in racial performance in IQ tests, asserts that IQ tests measure "intelligence", asserts that differences in IQ test performance are the result of genetic differences, asserts that difference in intelligence explains criminality, asserts that difference in intelligence explains or justifies differences in economic success and wealth.
This coalescing of views is the result of underlying ideological commitments; on one hand, to the idea that racial inequality in the United States is a blameworthy thing that should be addressed, and on the other, to the idea that racial inequality is the result of natural forces that do not need to be addressed.
The latter camp is going to border on, and in many cases contain, overt racism. If you're in a position where you have to filter the material being used to train your chat bot, the only plausible option is to throw out the racist parts of the source material. This is inevitably going to lead you to factual biases. There's no need to result to Mao quotes to explain what is happening here.
The unfortunate reality, of course, is that facts are more complicated than ideology. (It is a fact that there are racial differences in IQ test performance. It is NOT widely accepted that these differences are due to genetic makeup or that they explain crime rates.) But unless you want OpenAI to start picking and choosing individual facts to include in their training data, I don't see how they can do better than including everything, with a filter for racism, homophobia, etc.
It might only appear leftist if you're from the US, where you vote either for the extreme rightwing or slightly rightwing party.
From a European perspective many talking points and political efforts of the Democratic party are certainly extremely left-wing. In fact, many of the European left-wing parties crudely import talking points on internal US issues, like for instance Black Lives Matter. Also, why would a "slightly rightwing party" push for anti-white CRT in schools, sex transitioning of children, abortions and mass immigration?
Where, precisely, does OpenAI claim that ChatGPT is unbiased?
ChatGPT obviously has biases from its training data, and a separate set of biases from the filters OpenAI has (not at all secretly) placed over it to avoid content that OpenAI doesn’t want it to return.