The points you see are wrong, but they do have cell towers in the sea (often supplied by underwater cables). In this part of the world they can be found usually around the North Sea on or for the oil rigs mainly. However the majority of these points even on land are quite inaccurate. They are captured by simple triangulation - wardriving.

This is no cell towers exactly, this is the measurements related to dedicated cellID(sector if BaseStation(tower)). So such kind if issues is possible especially at water(rf signal propagation feature) and gps mapping if present.

I found something similar, finding a number of cell towers in a lake I'm pretty sure no one would ever put a tower in (it's about 10 feed deep).

I saw that too in, for example, Lower New York Harbor.