Lots of ifs, like if I installed a hardware keylogger onto your computer.

>I don't get it. You are going to connect to a "secure" server from an insecure machine with full access. Do you expect that your server stays secure after that?

Yes, XEN and hardware virtualization keeps it all safe. VMs like those on Qubes work the same way. Most laptops don't have good virtualization hardware, expandable ram, or decent processors. Connecting to a server doesn't compromise it in any real world scenarios, the same way if I remote access your laptop through mine, root ssh is not common, and you'll have it isolated in a VM anyway won't you?