Yup, IMAP is powerful, but it's gathered some baggage over the years.
Switchboard uses a simplified interface based on JMAP[1] for client's and workers, which it does proxy into IMAP commands. One of the conveniences of Switchboard is it manages the IMAP connections: restarting them as they fail and allowing concurrent access.
[1] http://jmap.io/