I use --single-instance and it spawns much faster (and eats way less memory). Even without this switch it does not take this long on my machine (certainly less than half a second), but yes it takes noticeably longer than, say, urxvt.