In some cases, avoiding heap allocation is just impossible. Take OCaml: last time I checked (over 10 years ago) there are two kinds of values: integers, and pointers to heap allocated objects. That’s basically it. Even floats are allocated on the heap (except when inside an array that’s the only optimisation). So unless you only do integer operations and avoid constructing any kind of data structure, you can’t avoid heap allocation.
Other languages are much more conservative than OCaml in this regard. That with value types and all. If they’re clear enough which constructs don’t trigger heap allocation, and the set of thereof is rich enough to allow significant work, then yes "just don’t use the GC" is a valid option. Not all languages (I mean implementations, but we can conflate the two in most cases) fulfil those two conditions. Maybe Go does.