My download has slowed to a crawl around 90% completion. I am doing 3k a second. When I read that your list was not sorted case sensitively I wondered what difference that may make.

I did a little experiment with the american-wordlist-insane wordlist and sorting in the interim. I used msort which does a case sensitive sort[1] and sort which does not. Here are the results:

    6830085 insane.msort
    1461684 insane.sort.xz
    1376548 insane.msort.xz

It is not a big difference but it is still a little better. For future releases you may want to do a case sensitive sort.

[1] Zebra comes before aardvark