Cool topic! This isn't surprising given the AI models would be trained such that existing medical practices, biases, and failures would propagate through them as others have said here.

There is a published, recognized bias against women and blacks (borrowing the literature term) specifically in medicine when it comes to pain assessment and treatment. Racism is a part of it but too simplistic. Most of us don't go to work trying to be horrible people. I was in a fly in community earlier this week for work where 80% of housing is subsidized social housing... so spit balling a bit... things like assumptions about rate of metabolizing medications being equal, assess to medication, culture and stoicism, dismissing concerts, and the broad effects of poverty/trauma/inter-generational trauma all must play a role in this.

For interest:

https://jamanetwork.com/journals/jamanetworkopen/fullarticle...

Overall, the authors found comparable ratings in Black and White participants’ perceptions of the patient-physician relationship across all three measures (...) Alternatively, the authors found significant racial differences in the pain-related outcomes, including higher pain intensity and greater back-related disability among Black participants compared with White participants (intensity mean: 7.1 vs 5.8; P < .001; disability mean: 15.8 vs 14.1; P < .001). The quality of the patient-physician relationship did not explain the association between participant race and the pain outcomes in the mediation analysis.

https://www.aamc.org/news/how-we-fail-black-patients-pain

(top line summary) Half of white medical trainees believe such myths as black people have thicker skin or less sensitive nerve endings than white people. An expert looks at how false notions and hidden biases fuel inadequate treatment of minorities’ pain.

And https://www.washingtonpost.com/wellness/interactive/2022/wom...