As an instrument-rated pilot, I reject your brazen assertion that radio work is "by far the hardest part of flying a plane." Are you speaking from personal experience, or do you have some data to back up this claim?

From an instrument proficiency point of view, I'm much more worried about efficiently understanding the information on the approach plate, maintaining situational awareness with respect to the approach path, adhering to minimums, and so on. Missed something that a controller said? "Say again." There's a reason it's aviate, navigate, and communicate--in that order.