It's not the hands that are the problem, it's the cognitive load. Hands free is no safer than hands on, when it comes to talking on the phone.
https://www.insurancejournal.com/news/national/2014/04/15/32...
https://www.insurancejournal.com/news/national/2014/04/15/32...
I don't see anyone advocating for a ban on conversation in cars.