Vertical mice need not be too expensive these days. OK, now I use Evoluent Vertical Left 4 (the company bought it for me but IIRC it's $90), but before that I used some barely-not-noname-brand for $25 and it was decent for the money! My biggest complaint is that buttons were a bit hard to press (i.e. you need more force comparing to Evoluent).

Now about keyboard+mouse combo... Yes, it's a bit awkward to carry Kinesis Advantage 2 + Evoluent mouse around. I actually don't do it unless absolutely necessary. If I would have to work from someplace for a few days, I would just carry laptop :)