Almost no one actually audits the source they download before they run it, whether it be a package installer, a configure/make/make-install tarball, or a curl|sh.
If you know enough to audit a shellscript, you can easily decompose the command-line to download, audit, and run the local copy.
For the vast majority that chooses to simply trust the author, there is no additional security issue with curl|sh compared to other non-signed distribution methods provided the script is wrapped in a function call or otherwise resistant to errors that might happen if a partial download is executed.
Should we do better than curl|sh? Yes. But it's no worse than downloading some random tarball or even cloning some random git repository, as long as it's served over HTTPS.