Have you tried https://github.com/openrq-team/OpenRQ or https://github.com/harmony-one/go-raptorq or https://github.com/cberner/raptorq (the last seems to have excellent decoding/encoding performance)?
(Raptor codes (a type of FEC) can essentially transfer over UDP at the underlying line rate, even if packet drop is high. In other words, if it is a 1 gbit/s link, with 5% packet loss, you will be able to send over 900 mbits/s without needing any TCP features). This is also helps high latency scenarios, as you don't need test/back off how much data to send...you just send as much Raptor encoded data as you want down the pipe, and as long as you send enough recovery packets with it, you will be able to reconstruct the original flawlessly).
(Also see https://par.nsf.gov/servlets/purl/10066600
http://www1.icsi.berkeley.edu/~pooja/HowUseRaptorQ.pdf
https://a1f9fb7d-b120-4c73-98e3-5cdb4ec8a2ab.filesusr.com/ug...