If there were populations with significant relict Neanderthal DNA, you'd be looking at more like the Celt/Gaul populations, and talking at least a thousand years before that -- think the British isles, before the Angles/Saxons moved in. And even that is very... speculative history.
Very speculative indeed, considering there was plenty of movement of people between the British Isles and the continent before the Anglo-Saxons arrived (who themselves had surprisingly little impact on the genetic makeup of the British people [0]). And if there was significant Neanderthal DNA in the British Isles 1500 years ago, it would still be detectable among living generations (it isn't).
"Neanderthal-derived DNA was found in the genome of all contemporary populations". [1]
[1] https://en.wikipedia.org/wiki/Interbreeding_between_archaic_...
See also: https://www.sciencemag.org/news/2019/01/genetic-data-half-mi... ( Genetic data on half a million Brits reveal ongoing evolution and Neanderthal legacy )
https://science.sciencemag.org/content/358/6363/655 : "We find that non-African populations outside Oceania carry between 1.8 and 2.6% Neandertal DNA"