"Today we think of the Amazon as a pristine rainforest, sparsely populated by indigenous communities. But in records from the 1500s, Spanish colonizers describe a densely populated region, crisscrossed by canals and sunken roads and dotted with bustling, fortified towns. Unfortunately, fortification couldn’t protect the towns’ inhabitants against European diseases, which devastated South American indigenous populations and left the fortified villages abandoned." https://arstechnica.com/science/2018/03/network-of-fortified...
"Some 5 million people may have lived in the Amazon region in AD 1500, divided between dense coastal settlements, such as that at Marajó, and inland dwellers. By 1900, the population had fallen to 1 million and by the early 1980s it was less than 200,000."
The New World people only domesticated a handful of animals, such as the guinea pig and the llama. So, fewer opportunities to catch something from them.
Still, they had a new disease that spread to the Old World: syphilis, which certainly killed several million Old-Worlders, only it took longer to run its course and could be prevented by, well, chastity.
An interesting idea: if you count victims of smoking towards the total death toll (tobacco is a New World plant), it may well be that the Old World suffered more deaths from the Columbian Exchange. But spread over a much larger population and a much longer time, these deaths didn't amount to destruction of entire civilizations.
See https://en.wikipedia.org/wiki/Native_American_disease_and_ep...
And a huge number from forced labour, executions, and torture, well described by the colonizers themselves.
Bear in mind that the slaves weren't all black, and the slavers weren't all white.