I grew up in Australia, so would have considered myself a native English speaker. We’re even a British colony!

I moved to Scotland for a year, and London for a few years later. I’d regularly here phrasing I’d never heard before and would need the speaker to clarify or would look up later. In the vast majority of those cases it was most often an old word that appears to have fallen out of fashionable usage. “Ken” being a synonym for “know” in Scotland being the one I remember most given how frequently it was used.

So it’s not just non-native speakers that struggle at times!