With long-polling you have reconnect after every message, which is a very expensive operation (relatively speaking). HTTP headers often form the bulk of the request, and the number of connections you have to maintain is high. All of these things add up.

In addition, TCP slow start, window sizing, etc, work against you when it comes to optimizing for latency: http://www.igvita.com/2011/10/20/faster-web-vs-tcp-slow-star...