Exactly. This is why it's important not just to have language skills when creating these kinds of datasets, but also cultural knowledge and context.
For example, to pick a bit on the Google Emotions dataset again, it's difficult to label this message...
“Also Republicanism is a belief system. It’s taught and handed down like religion. Conservative talk radio is its evangelism.”
...unless you're familiar with US politics. Hence why it was labeled as APPROVAL by the non-US annotators, even though it's criticizing Republicans.