That said, definitely in the case of hotels there are loads of hotels it seems there is a legal aspect as well. If you lived in China you might be familiar with the long and boring process of registration with the police every time you move house. Sometimes you even have to register at more than one place because you moved into a different urban area. According to the law, all foreigners must do this everywhere they go in China, including tourists who are on holiday. It wouldn't be very efficient to make tourists spend a couple hours at the police station every time they checked into a hotel, so some hotels have some arrangement with the local police to do this on their guests' behalf behind the scenes. Most tourists nowadays don't even know that they have been registered with the police everywhere they travel. But smaller hotels don't provide that service. I have successfully blagged my way into staying at one smaller hotel by assuring them i would visit the local police myself to register, but a lot of them don't want the hassle or (perhaps) attention from the local authorities, so they just ban foreigners.
When I left China it was during the pandemic, and pretty much no hotels at all would allow foreigners to stay. This was xenophobia, pure and simple. There were a few times I had to first visit the police station and ask the police where I could stay, because hotel after hotel after hotel refused me, despite my green health code, valid visa and so on.
Also I was always greeted with utter friendliness, so I don't get the everybody's xenophobic aspect. I do speak some (very basic) Mandarin though so maybe that helps with how you're treated.
It is true that people in China tend to be fairly polite, although when it blows up it tends to go straight to 11. (I saw a lot of this happening between police and residents during the coronavirus lockdowns.) But just because people are polite, don't assume that means they're not bigots. In some parts of the country, particularly amongst more affluent people, there is a sort of attitude that foreigners should be treated with kid gloves, which is thoroughly degrading.
Needless to say, I loved it.
Also, I don't know what makes you think that Chengdu is off the beaten path - it's a massive tourist destination, in particular because of its panda reserve, presumed proximity to Tibet and the global renown of Sichuan cuisine (home of mapo tofu, hotpot etc).
Thirdly, this coddling of foreigners is exactly the kind of racist behavior that I find to be degrading. It might feel superficially nice to be treated like you are special or unique, but really it means you are not being treated with respect. They are treating you like a child, or a curiosity. If you visit some of the indigenous communities of China then you will often see the Han majority treat the ethnic minorities in the same way, as if they are just props for photos or some kind of weird creatures to be gawked at. It's gross, imo.
It’s definitely taken too far sometimes though.
Try go to the country side to see the real China.
For the racism part, I wouldn't say that signs with "no foreigners" in shops are common, but I have seen them a couple of times during my 5 year stay in China.
Familiarity breeds contempt.
Western expats in East Asia tend to be quite polarized about their country of residence, and expats in poorer countries even more so. Over time, the tint of novelty wears off and the warts begin to stand out. Poorer countries have more warts.
For folks like myself who are extra sensitive, the negatives get an outsized representation, while the positives and neutrals get filtered out. It took me years of to develop the habits to compensate. I'm far from where I'd like to be, but I'm learning to accept that as well.
This is especially true in places of higher density. If you encounter 1 bad apple in a place of 100 people, vs 10 bad apples of 1000 people, the ratio is the same, but subjectively the latter feels ten times worse. It’s the price you pay for living the city life.
And when you have an under-stimulated career, the idle mind becomes the devil's playground.
We let collective narratives plays a greater role in colouring our opinions (as opposite to direct experience) than we'd like to admit. In this day and age, I don't think it's especially controversial to say that we get more dopamine hits from internet discussions than having a stroll down the street. Ultimately, unless we consciously intervene, the chemicals get to decide what we let ruminate in the back of our minds.
The idiosyncrasies you used to brush off or find amusing are now small but cumulative signs of impending doom. What we get right in direction we get wrong in magnitude. The sprinkles of verifiable truth can often as easily fuel our biases as they moderate them.
It seems to get worse every year also. My first visit in 1999, foreigners were quite welcomed. Then it simply got more mundane from there. My visit twas chengdu in 2006 was fairly like you stated though.
Most people are still ok, but some fraction of the population has a mixture of fear and hatred of foreigners. They think foreigners are dangerous because we bring the virus. Far fewer places accept foreigners than before.
I love my life and my friends here, but walking down the street in a village and having a guy wearing an official-looking coat scream at you to “fuck off” has a way of souring one’s mood.
Edit: to be clear, I'm not trying to deny your own experience or making any accusations here. I'm just presenting the other side and clarifying that I don't feel it's a universal experience.
The era of Northern Europeans ("whites") dominating the globe is over forever, so no need to beat yourself up about it. Just letting you know that this extreme narrative is quite bizarre to non-Americans.