The nice thing about C# is that, unlike Java, you actually get all the low-level things. They're just tucked away, out of sight of your average coder who's more likely to shoot themselves in the foot with them. But if you know how to use them, it's all there. Let me enumerate.
C# has value types, which follow the same memory model as in C (i.e. stack-allocated for locals, embedded directly into the outer object as fields). You can request explicit layout mode, whereby you can set an offset for every field manually - if you set them all to zero, you get a C union. Alternatively, you can request automatic layout mode, where the JIT is allowed to reorder the fields in arbitrary ways to optimize memory and/or access, which is something you don't even get in C.
It has raw pointers. Unlike object references, these have the usual C semantics - there's no GC involved there, and you're responsible for keeping the objects alive, so you can have dangling pointers etc. No boundary or null checks, either - it's zero overhead. You can do pointer arithmetic on them, including indexing with []. If you P/Invoke malloc or equivalent, this gives you C-style heap allocated arrays.
It has stackalloc, which is a language operator that's equivalent to the non-standard alloca() function in C - allocate a chunk of memory on the stack, and return a pointer. This immediately gives you stack-allocated arrays as flexible as C99 VLAs.
With generics, if a generic type parameter is a value type, the specialization for that type is separately JIT-compiled and optimized. This can be used in a way very similar to C++ templates, for zero-overhead inlined callbacks and similar shenanigans.
All in all, if you want to write high-perf code in C#, you certainly can. You'll hit the limits of the JIT optimizer soon enough - it can't be as good as a full-fledged AOT C++ compiler, say - but you can certainly shed most of the overhead associated with a managed language.