Modem sounds were totally useful for troubleshooting. You'd know if your modem dialed someone's voice line, or answering machine, or a fax machine, or a busy signal, or "number not in service", or if it just rang with nothing answering, or if someone was already using the line for a voice call, or blessedly if it hooked up with an actual modem on the other end. You could also hear interruptions from someone else picking up a phone on the line or from call waiting; not all modem hardware would do it, but the relay on mine would audibly pulse.
The sounds of the actual connection weren't all that useful, but did at least indicate the negotiated speed between the modems (28800 and up does the kabing-kabing sound, 14400 and lower doesn't). So you'd know quickly if you hit a slow node on your BBS or AOL.