As you said, all contemporary populations. So there is nothing to suggest that the British Isles was a Neanderthal genetic time capsule prior to occupation by the Anglo-Saxons, Romans, or Celts. Likely all of these incoming populations were equally as Neanderthal as the existing inhabitants.
"Neanderthal-derived DNA was found in the genome of all contemporary populations". [1]
[1] https://en.wikipedia.org/wiki/Interbreeding_between_archaic_...
See also: https://www.sciencemag.org/news/2019/01/genetic-data-half-mi... ( Genetic data on half a million Brits reveal ongoing evolution and Neanderthal legacy )
https://science.sciencemag.org/content/358/6363/655 : "We find that non-African populations outside Oceania carry between 1.8 and 2.6% Neandertal DNA"