People do consistently audit unsafe usage. And I would rather have atrocious rust code that's memory safe, than atrocious C/C++ code that comes with a free package of CVEs.
There are tools that explicitly exist for this use case, such as cargo-geiger [0]. There was some drama with a large framework called Actix a while ago due to the maintainer having a bit of a cavalier attitude towards unsafe usage. Etc.