I think some people simply aren't aware that "G**y" references the Roma (and is also considered a slur by most), because they associate it with the negative stereotypes which have been ascribed to them.
Most often it is the mainstream society living in stationary houses describing people living in temporary encampments with a different culture.