I find that manual assembly is mostly useful when I know something about the data or desired operation that is not possible to communicate to the compiler.
But sometimes it is! To many, however, it can be difficult to know when.
Even more difficult: for how long? - Compilers get better and CPUs get new instructions one might miss in the manual code. If you have your tight critical loop in assembler one has to regularly verify.
I feel that if you're one of the rare people who know how and when crafted assembly works better, it's almost always also a better use of your time to work on the compiler, rather than whatever you are working on.
Even years ago when the compilers were not as good this is exactly what most people did. Compile it look at the asm. Sometimes see better bits here and there and replace that function with an asm one. If you are in a space where that perf really maters you do this more often.
But it doesn't happen often these days, does it?
In cases of parallel processing where code executing in the critical path is a significant bottleneck, hand-rolled SIMD can sometimes outperform what gcc or clang spits out by a few clock cycles/iteration. But even then, the sanest way to do that is to let the compiler do most of the legwork and then incrementally tweak its output.
or, use a GPU.
IME gcc is extraordinarily bad at anything but the most basic of auto-vectorizations, but its output from SIMD intrinsics is decent -- not usually quite as good as hand-rolled assembly, but much faster to write (for me anyway, YMMV) and still within 2x performance.