Yes, they’d panic without training and practice. Because people panic, get frantic, or otherwise become emotional to the point of misunderstanding. Calm communication in serious, dangerous, or emergency situations is learned.
But other people who use radios, like the Army, don't speak in that drawl but also don't panic on the radio, and were doing it well before.
It's not the specific way of speaking that prevents panic. It's the training. It just happens that for a while pilots all wanted to sound like Chuck Yeager and then it just became the way new pilots talked because that's how all the other pilots they knew talked.