My experience with optimizing compilers is that generated code is often frustratingly close to optimal (given the source is well written and taking account the constraints of the target arch).
It is perfectly reasonable to take a look at the output on Godbolt, tweak it a bit and call it a day.
Maintaining a full assembly language version of the same code is rarely justifiable.
And yet, I understand the itch, especially because there are quite often some low-hanging fruits to grab.