The process can be slower if the retailer uses an older system that hasn't got a permanent network connection. With some ancient terminals, you could hear a modem dialing up and talking to the bank. That was slow, but luckily those kind of terminals are now rare.
When using a chip, my process is: insert card, wait for transaction to complete, remove card, put everything away, go.
You'll notice that putting stuff away happens in parallel with the transaction in the first case, but they're in series in the second case.
It may be reasonably quick, but it's still noticeably slower than swiping. It's not a big deal, but it's just one of those little annoyances.
Edit: Swiping used to take 1-2 seconds to process, EMV seems to take about 10.
Worse yet, not all the chip enabled readers have the chip part enabled. So you can stick your card in, wait, and then realize you have to swipe. Otherwise, you swipe, get the suprisingly aggressive BZZZRT sound, and then use the chip reader and wait at least 5 seconds while it authenticates.
I'll time it at the local Target. It's slow.
In terms of speed from fastest to slowest: Apple/Adroid, swipe, swipe and sign, swipe and PIN, dip and sign.