Inferred return types don't make refactoring easier. In my experience they make it much more difficult, because you have to look at function implementation to understand what it does. Particularly for error handling, you always want to see what errors a function may return even in library code.
And all that said, there's no distinction to me between internals and externally facing code when it comes to quality and standards. If you have a function, even if it is called once, it should always have its return type annotated. It is more correct than leaving it to be inferred by the compiler, because you are explicitly annotating the intended behavior of the abstraction. This makes it easier to write correct code and for verifying it by a reviewer.