With Japan and Korea (especially the latter) towards Americans, there's also a soft-unspoken rule (that sort of goes both ways) due to the relationship those countries have fostered towards each other. A Brit/German/Italian/etc who spends more than a short visit in Korea/speaks Korean will probably start being taken to the side for flouting cultural norms like age-deference, polite speech, etc to be informed of their cultural mores (usually phrased with an indication that they also come from a structured society, they should understand that this is the way it is); while this will rarely happen to the same group of Americans. In some cases it's the "dumb/naive American" effect, but it also has to do with the larger relationship between the two countries.
Less importantly but somewhat obvious: many Koreans are fairly competent in English and familiar with common American accents. They’ll know pretty quickly if you aren’t speaking with one.