The New World people only domesticated a handful of animals, such as the guinea pig and the llama. So, fewer opportunities to catch something from them.
Still, they had a new disease that spread to the Old World: syphilis, which certainly killed several million Old-Worlders, only it took longer to run its course and could be prevented by, well, chastity.
An interesting idea: if you count victims of smoking towards the total death toll (tobacco is a New World plant), it may well be that the Old World suffered more deaths from the Columbian Exchange. But spread over a much larger population and a much longer time, these deaths didn't amount to destruction of entire civilizations.
See https://en.wikipedia.org/wiki/Native_American_disease_and_ep...