I intentionally avoid this, especially if I am not the only one connecting to a server. I see potentially breaking others' expectations for my convenience as a clear downside. Plus there is a level of paranoia there, I might have unintentionally included a credential in some config that gets copied over. That potentially increases the blast radius of a server compromise beyond what is strictly required for that specific server. I might copy over some scripts into a directory not in $PATH, easy enough to add it to my shell session if needed, CTRL+R helps as well.