Yes and no. Yes, almost all of the standard library collection are allocation heavy and it is still the dominate pattern in C#, so if you want to avoid the GC you need to avoid these and resort to building your own primitives based on Memory/Span. Which sucks.
However, you can use interfaces in a no GC world since you can constrain those interfaces to be structs or ref-structs and the compiler will enforce rules that prevent them from being boxed onto the GC heap.
Also of recent note, the JIT can now automagically convert simple gc-heap allocations into stack allocations if it can trivially prove they don't escape the stack context.
> It would be better if the GC can be turned off with a switch and just add a delete operator to manually free memory.
It is a little know fact that you can actually swap out the GC of the runtime. So you could plug in a null implementation that never collects (at your own peril...)
As for a delete operator, you can just roll your own struct based allocation framework that uses IDisposable to reclaim memory. But then you need to deal with all the traditional bugs like use-after-free and double-free and the like.
For me, I think low-gc is the happy medium. Avoid the heap in 99% of cases but let the GC keep things air tight