People from the Nordics had trade relations with Southern Europe and Asia all the way back in the bronze age. Alas, records are vague at best, but there's definitely reason to believe there was an influx of people and ideas from the Black Sea area before and during the migration period, just a few hundred years before the viking raids and Norse settlements all over Europe.

And as if that wasn't enough, the Norse were quite infamous for taking captured women as wives and they also took slaves from the peoples they raided. Those people didn't just dissapear from the gene record.