As for your claim, you're mostly right. There was an incursion of non-African DNA into Subsaharan Africa around 3000kya, probably related to Bronze Age Collapse and migrations around it. Because of this, Africans can carry up to 25% Eurasian DNA, and part of that is Neandertal.
That seems to be what happened with Neanderthals, at least at first.
Incidentally any violence may have been from the Neanderthals because almost all of the genetic data shows Neanderthal ancestry from Neanderthal males and homo sapiens females, and not the other way around.
Even when this did happen[1], the record seems to show trickles of gene-flow from these populations entering a much larger stream of genes from a more recently out-of-Africa population.
[1] And in a big complex world, it certainly would have happened sometimes.