Yes, I am sure that westerners are saying ghee wrong. The sounds they make when they say "ghee" maps to a completely different (non-existent) word in Indian languages

It's the exact same way Indians mispronounce many English words when they map it to some equivalent native sounds.