The French and their globally disconnected departments ...
(French Guiana is governed by EU regulations, after all...)
Or to be a bit more precise, the rule says any "European state" can join, but there are no set-in-stone definitions of what that actually means.
People here are mentioning Morocco as the example of a country rejected for not being European, but that's not quite accurate: Morocco got rejected from the European Communities in 1987, six years the Maastricht Treaty officially established the EU. Maastricht Treaty specifically put the European Commission in charge of figuring out which countries are considered to be "European", but as far as I know, they never said no to anyone. They did say yes to Cyprus, Georgia and Armenia.