Nobody claims they all speak Finnish well, but the Swedish spoken in Finland is influenced by Finnish and typically includes some loans from Finnish.
For example, in a bilingual environment, it can be enough to understand two languages and to speak one.