Edit, for other's curious, as the person I'm replying to said, it's not racist per se, but has its origins in racist practice:
> The term originated in late nineteenth-century legislation and constitutional amendments passed by a number of U.S. Southern states, which created new requirements for literacy tests, payment of poll taxes, and/or residency and property restrictions to register to vote. States in some cases exempted those whose ancestors (grandfathers) had the right to vote before the American Civil War, or as of a particular date, from such requirements. The intent and effect of such rules was to prevent African-American former slaves and their descendants from voting, but without denying poor and illiterate whites the right to vote.
I often hear people use the phrase "call a spade a spade", thinking it's saying "if a card shows a spade, call it as a spade (when showing the hand)", but I've heard for a long time that phrase is also racist. Not growing up around anyone who ever used that phrase, I would have had no idea.
Makes me wonder how many other phrases out there in every day use don't have a happy meaning, even though at first glance they might seem innocuous.
The spade is question is a digging tool.
But "chink" is also an offensive term for a Chinese person, so "chink in one's armor" can be misunderstood, especially if what you are talking about actually involves a Chinese person. ESPN got in trouble a couple of times with it, even though they had used it thousands of times before without incident, because of a couple articles where the weakness they were describing on some team was a player who was Chinese.