The issue with libraries which offer you portable simd is that the auto vectorizer of the compiler will likely generate faster code.
For example, consider you want to compute the average of a list of floats. The compiler cannot autovectorize this, because float addition is not commutative. However, it's much faster to do component-wise addition in groups, then a horizontal sum at the end, and then divide. Whether it matters depends on your use case, and the compiler unfortunately can't read your mind, so it has to be conservative.
Gcc and llvm can tell you if they can’t Auto vec a function, maybe rust could turn this into an error at comptime.