The hypothesis of why local conversations are safer is that when traffic gets complex, the passenger will see that as well and pause the conversation. The person on the phone doesn't have that awareness. However some jurisdictions have restrictions on passengers for younger/provisional drivers, for just that reason.