> would this have been a problem had it been written in rust?

The answer would be "it depends on whether you consider denial-of-service a problem". The key detail which makes all the difference is that, unless you're playing with raw pointers (which can only be dereferenced in "unsafe" blocks), the pointer to a buffer slice is always kept together with its length (in a "fat pointer"). Attempting to write through it past the buffer's bounds will result in a Rust panic, which usually aborts the whole process (there are ways to abort just one thread, or even to treat it similarly to a C++ exception, but a library cannot depend on them since the program might be compiled in the panic=abort mode). While that's obviously better than allowing for remote code execution, it still could be considered an issue.

Of course, that's assuming you want a similar API and are writing the code in a similar style, just replacing the direct pointer manipulation with slice manipulation. I don't know whether, in this particular case, more idiomatic Rust code would have avoided the issue. And, of course, the "growable container" approach would completely avoid it, but Rust is also used in places where memory allocation is not allowed, so having a non-allocating API still makes sense.

Rust would take the growable container approach here, so it wouldn't be vulnerable in the same way. There's nothing in C that prevents you from doing this either, but as the parent post mentions it's common style to do this (see also things like snprintf).

In no other mainstream language other than C (and maybe non-idiomatic C++) would this have ever been a problem.

It is an exercise in navel gazing until an organization steps up to fund: 1) a linkable clib harness that exports Rust functions to replace OpenSSL functions. and 2) pays to get the resulting harness and underlying code FIPS certified so people can use it.

Rust code would probably return an error on output buffer overflow instead of siltently discarding data. Something like this: https://docs.rs/base16ct/latest/base16ct/mixed/fn.decode.htm...

Probably not. It's definitely possible to write this function exactly as it is in openssl with this issue, nothing about Rust's safety claims prevent it. However, Rust doesn't have the same issues that lead to this kind of pattern being common like they are in C. Result<> types being so common, better handling of dynamic allocations, and at least for now a generally (it can't last forever) more security conscious community mean that other patterns will be chosen unstead. I'm not sure any language can actually prevent it from being possible outright, but you possibly pair it with some kind of formal proof system like using Coq and TLA+ might let you prove thag an issue like this doesn't exist in a specific implementation. However it's incredibly hard to do this.

Not in this case as far as I can tell.