Very speculative indeed, considering there was plenty of movement of people between the British Isles and the continent before the Anglo-Saxons arrived (who themselves had surprisingly little impact on the genetic makeup of the British people [0]). And if there was significant Neanderthal DNA in the British Isles 1500 years ago, it would still be detectable among living generations (it isn't).