* The predominant oceanic currents in the South Pacific around Rapa Nui come from Polynesia, not South America. So you're not likely to be drifting towards Rapa Nui. Indeed, traveling from Polynesia to South America via Rapa Nui isn't a particularly far-fetched idea.
* Polynesia, and earlier Austronesian antecedents, had a much stronger tradition of long transoceanic voyages. (Austronesians crossed the Indian Ocean and settled Madagascar before Africans set sail from their own coast to reach it).
[0] https://www.sciencemag.org/news/2015/07/mysterious-link-emer...
I would put my money on the islanders being the explorers rather than the mainlanders.
(Writing this I realize you might have meant that people from South America went to Easter Island and back, but as far as I know there hasn't been found traces of a strong culture for navigation in South America.)
Further, the motives for diaspora were not there. Native south Americans had barely scratched the Amazon. There was so much left to explore and exploit on the continent. Going on a crazy at sea journey just was not necessary.
The Amazon basin had millions of inhabitants in the 15th century. Population declined precipitously after European diseases arrived, and did not recover to prior levels for 300+ years afterward.