The entirety of Rust's `std::cell` is a confession that yes, we really do need mutable aliases. We just pretend they the aliases aren't mutable except for a nanosecond around the actual mutation.
Controlled mutable aliasing is fine. Uncontrolled is dangerous.
Except that's a really harsh and unproductive way to phrase it. The existence of "unsafe" is not a "confession" that safe code is impossible so why should even try.
Progress in safety is still made when an unsafe capability is removed from the normal, common, suggested way to do things and it is moved to something you need to ask for and take responsibility for, and hemmed in by other safe constructs around it.