"When the programmers get it (the specs), they often start negotiating with him about what's going to be built. They exchange countless emails, and he's being constantly interrupted and being asked to clarify exactly what the spec means. "
A simple solution is: speaking about the project/product feature with your developers first. Understand what will take long, what won't, what can be stripped out in the first iteration. Once this is done, work with designers to push something out.
Another super simple solution is videos.
Writing takes time. Wherever possible I try to make videos for the designers or even better have live calls.
In terms of idle time, there should be always a backlog of specced out work or bug fixes that the developers can focus on. Failing that, they can work on optimising past projects, refactoring or reading about new things.