Canada has a sea border with France.
https://en.wikipedia.org/wiki/Hans_Island
Also, I think the constraint to be in the EU is that it has to be "an European country". It's relatively vague.
Someone else pointed out Canada also has a land border with Denmark: https://news.ycombinator.com/item?id=49741686
What about Australia? New Zealand? Japan? Maybe Indonesia (I don't know maybe it has something on the Asian mainland).