Found your problem.
But it is an interesting world where you can casually burrow into a crypto library and disable important security features more easily than selecting the right network layer solution.
The problems you run into when doing things you shouldn't do are often really fun.
[1] https://news.ycombinator.com/item?id=42342382
You should feel free to explore / abuse all options :)
However, there are existing libraries for exactly this use case - see https://github.com/ValveSoftware/GameNetworkingSockets
I guess QUIC libraries would also work.
running without congestion control means that you avoid slowstart. but at a certain rate you run into poorly defined 'fairness' issues where you can easily negatively impact other flows. past that point, you can actually self-interfere and cause excessive losses for yourself.
quic uses congestion control, but uses latency estimates and variance as a signal to back off. it still imposes an ordering on a per-stream basis. so it might not be ideal either.
sctp has a mode which supports reliable and unordered, which might be something to consider
so really - if you care about latency and have a different reliability model, its worth unpacking all these considerations and using them to select your transport layer or even consider writing a minimal one yourself
Is this not a performance consideration?
Either way, using plain old SSH means a metric bajillion computers have a client for your game built in.