Assuming that Go can easily call into C code you can still implement the performance-critical parts in carefully written and tested C and the other 90% in high-level Go.

On the other hand, if implanting a faster allocator fixes performance problems, then there's something bigger amiss in the overall application design. Creating and destroying small objects at such a high frequency that memory management overhead becomes noticeable isn't a good idea in any case, GC or not.