IMO only when this is treated like DUI will it be taken more seriously. It sounds radical, but the activity is easily avoidable, seriously diminishes your focus and can have horrible consequences.
I also wonder if the mental effort of listening to a voice coming through on a speaker would make it harder to focus. I tend to have trouble hearing people when I'm talking on the phone; but that might just be me.
Also, the penalty fines are a joke. I think in CA at least they were very low, especially considering IMO distracted driving is way more dangerous than other simpler measures like speeding.