From GPT-2 to GPT-4: How LLMs understand different occupations over time

These results are largely uninterpretable. Leaving aside minor nitpicks like many of these prompts being barely grammatical and highly unnatural ("The [x] person works as a slave"? you don't 'work as a slave', you are a slave), the comparison of GPT-2 to 3.5/4 conflates a massive leap in scale with RLHF tuning. It is possible the differences here are mostly due to the RLHF tuning, and the ones she pulls out generally seem like noble lies raters might be engaged in:

"However, for a subset of the occupations, the shift was made clear when comparing proportional changes from GPT-2 to GPT-3.5 to GPT-4. The newer models tended to overcorrect and over-exaggerate gender, racial, or political associations for certain occupations. This was seen in how:

    Software engineers were predominately associated with men by GPT-2, but with women by GPT-4.

    Software engineers were associated with each race mostly equally by GPT-2, but mostly with Black and Asian workers by GPT-4.

    GPT-2 exhibited an associated between the religion and working in a religious profession; GPT-3.5 and GPT-4 exaggerated this association manyfold.

    Politicians and bankers were predominately associated with liberal people by GPT-2, but with conservative people by GPT-4.

These patterns became more pronounced when compared with U.S. Census Bureau data, particularly for software engineers.

I am not advocating for language model outputs to perfectly mirror real-world occupation distributions. In fact, promoting increased representation in media for jobs traditionally dominated by one gender, such as nursing or engineering, is crucial for challenging stereotypes."