It would be scp, not sftp, but I often don't know what I'm about to edit until I get in there and dig around.
So, if "digging around" means looking at folders' structure, you can do it through SFTP without copying everything (unlike scp or rsync).
And if digging around means running commands, nothing prevents you from having a session for running commands open in parallel.
In any case, it's literally been years since I've had to.