Metadata leakage aside: If you run a parser on unauthenticated input, bugs in your parser can be exploited. This is partly why mac-then-encrypt is a bad idea (you have to decrypt to verify the MAC.)
Maybe people hand-roll their own signature validation code badly, but those same people will just as much screw up or plain disable the CA verification.
If you have the knowledge to use the primitives from something like NaCl, you have nothing to gain from using a full TLS stack but pain building, upgrading, programming your firmware and massive middleware problems in the field. And when they inevitably find issues in the TLS stack you used, your device is now fucked since there is no virtual address space, W^X, even stack cookies..
[citation needed]