First, most of the Viet-Cantonese words are likely brought by Chinese migration since the 15th century. When the Ming dynasty fell, many Chinese fled to South Vietnam and helped Viet people annex that land from the Cambodians https://en.wikipedia.org/wiki/Hoa_people
Second, tt's true that we "might" have lots of connections with the ancient Yue people, but this is still in debate. Just like the Romans called everyone on the other side of the Rhine "Germans", the Chinese of 2000 years ago called everyone south of them "Yue", even though they can be different tribes or nations. Being named the same thing by the Chinese doesn't actually mean anything. Genetically we have many similarities with the people of south China, but the history of vocabularies can be more complicated than that.