No, I was talking about SFTP, not scp. OpenSSH server has SFTP enabled by default. You can mount it using sshfs or browse it using any SFTP client (or any program with embedded SFTP client, which is most IDEs).

So, if "digging around" means looking at folders' structure, you can do it through SFTP without copying everything (unlike scp or rsync).

And if digging around means running commands, nothing prevents you from having a session for running commands open in parallel.