This is probably the smartest move. When I am dealing with someone who doesn't seem to understand what I am saying due to language barrier, I start dealing with physical communication instead however possible. It can look odd, and be sort of funny to witness; but it works a charm more often than not. There are just some things we all generally understand once the body becomes involved.

Two confusing ones I hear are:

“It’s my right, your left” (not common but has occurred multiple times).

The more frequent one is where someone is trying to say “medial side of left leg” and say “it’s the right side of of my left leg”.

If all else fails the penis points to the problem.