We use aho-corasick to extract keywords from terabyte size corpus. Works really well but you have to build your matching tree before.