I originally went the same route as you, and found that https://github.com/pion/webrtc is probably the best package out there for webrtc. I learned go just for it, and it paid off tenfold. Less memory, more connections, lower latency.
We got super lucky that a company heavily depended on SCTP in the early days. One of their devs (enobufs) did an amazing job with it.
Its ability to be simultaneously low-level and easy to work with is truly something else.