It's already solved (by humans) for Java, which can now be used for HFT. It seems like it's possible to do for C#.
You can write only-stack-alloc or limited-alloc C#, and Microsoft have put a lot of work into it (Span etc); it's just a bit unidiomatic.
Mind you, the last time I had contact with HFT it was inside an FPGA context..