To give you an example of what you can do I often use tramp locally to edit files as root: instead of having to fire a different instance of an editor as root to edit a config file I can just do it "remotely" from my main Emacs instance, with all my config and without worrying about messing something up by having the entire editor running privileged.
I work in embedded development so I also often use it to browse and edit files on the live target system (beats using a crappy dumbed down vi from busybox). And of course as you mention for general sysadmin tasks it's also great, I'm a developer but that doesn't mean that I don't have some sysadmin to do from time to time.
My current solution is just SSH/Vim but I think I'd like to go back to a graphical editor. I like Vim but sometimes it's a hassle on the big projects.