I think you are mostly right, but I've heard this effect observed elsewhere.
I recall a blog post about TurboTax or some other tax software. The part where it says "checking your returns" -- if it takes longer, people view it as having "done something".
You can probably check a tax return, which is really about 100 numbers, or maybe 10,000, in less than a millisecond. But it's better if takes like 10 seconds.
Someone might know what I'm talking about and have a link ...
I do think it's partially the latency, but the UI concerns you pointed out are probably more important in this case.