The immutability prevents entire classes of bugs, and helps enforce predictability on the part of your code. It also is assumed at least in part when it comes to message passing (else, what happens if you change a referenced binary after passing it in a message? Does the message change too, or remain static? Either way, complexity)
The message passing paradigm for interprocess communication ensures that the developer is forced to ask "What happens if this isn't received", i.e., if the process is down or missing or etc. Any sort of 'return', via a message sent back, could not happen; this is akin to a called function throwing, which again, requires the developer to ask 'what happens if we never get this response back'? The answer is never to wait indefinitely, it's to...what? Other languages make it very easy to ignore possible exceptions; in Erlang, not only are they likely handled (if you have your supervisor structure in place), but you're forced to decide how such a thing in another process affects the current process (if at all).
Because of the message passing paradigm, the distribution story is simplified; it's largely transparent as to whether the process you're sending a message to is local, or remote. This allows you to build in redundancy across nodes without many of the complexities (and thus, room for errors) that other languages give you, nor the lying abstractions many languages give you (such as RMI).
And having the distribution Erlang does helps with the reliability side of things, as it makes it comparatively straightforward (still complex, but far less so than most other languages) to get solid handling in the event of machine failure.