Certified brown person here. The only thing I think about in terms of deciding where to stop is how far I can get before needing gas, and what the odds of finding decent food are. I've travelled in rural areas all over the country: midwest (my wife grew up in rural iowa), west coast (wife's family is from the rural oregon coast), and south (worked a summer in southern virginia, my best friend lived in south georgia for a decade). I just got back from a road trip through rural Utah, Idaho, and Wyoming with my white wife, mixed kids, and Latina au pair. In all this time nobody has even looked at me sideways.
Hell, the precinct where I live went for Trump 58-34 in 2016 (the year I moved here). The precinct a few minutes away where my parents live was 57-32. Most of the ones around us were 60-30. Again, no problems.
I have to agree with the sibling comment. I have no idea how these folks would perform on an IAT (and I don't care because they're bad science: https://qz.com/1144504/the-world-is-relying-on-a-flawed-psyc...). Southerners are nice to visitors and keep their thoughts to themselves for the most part. I'd rather deal with that than west coast frigidity or NYC aggressiveness.