But, again, you can remove 80-90% of it if you don't care about compat. People run into a cognitive block when they think about TLS because they assume TLS means "browser compatible". But no browser speaks a NaCL transport today, so that's out the window.
Without the requirement to support browsers, you can:
* Allow only an ECDH handshake.
* Allow only the Chapoly ciphersuite, in TLS 1.2's AEAD format.
* Eliminate CAs and do an SSH-style key-continuity scheme.
This TLS "subprotocol" already exists and is already supported by most of the TLS libraries, all of which have been audited. It's supported by middleboxes and monitoring tools so it can be deployed operationally. Every mainstream programming language has bindings to it. Meanwhile, NaCL doesn't actually provide a transport protocol or even the security semantics of a secure transport; that's work you'll have to do de novo, and you will generate bugs doing it.
So what's the advantage to scrapping TLS and redoing it with NaCL?