The biggest issue with "set -e" is that it's inconsistent. It doesn't always cause the shell to exit where you'd expect it to, and in some cases, it causes the shell to exit where you would NOT expect it to. This is the most surprising:
set -e
i=0
(( i++ )) # shell will exit here
Still, I tend to use it because even though it doesn't handle every case, I'd still usually rather my scripts exit when a command fails.