Just use a non-graphical ssh session if you just want a shell on the remote machine