Good tip on the SDE too. I had heard of it but didn't realize it was so lightweight, dynamically patching the binary to emulate only the needed instructions. Cool!
Please take look at https://github.com/WojciechMula/toys/tree/master/avx512-remo..., I put there some raw numbers; will update the article later.
For English texts your algorithm is more than 20 times faster (twenty, not a typo), for CSVs the speed-up is "only" 9. However, I'd love to compare it with a well tuned scalar code.
One important take away for me is that I underestimated pext; it's a powerful instruction. (on AMDs terribly slow, though)
I'm still thinking about this one. It seems that ~.5 cycles/byte is slower than it should be. There's 6 pext->kmovq->vpaddb chains, where the pexts and kmovqs are all independent and can be pipelined. One chain should be 3+2+1 cycles, so all 6 should be 11 cycles. Agner Fog's site doesn't have information for vpermb yet, but a Stack Overflow comment says it's 1 cycle (which is rather surprising, that's 64 64->1 8 bit muxes--lots of silicon!). The latency on most other instructions can be hidden.
I updated my code with a couple minor optimizations, that at least lead to some nicer assembly output:
Getting rid of the if (mask) branch--it should almost never be false in real code (and never is in the microbenchmarks).
Inverting the mask during the comparison:
uint64_t mask = _mm512_cmpneq_epi8_mask(input, spaces)
& _mm512_cmpneq_epi8_mask(input, NL)
& _mm512_cmpneq_epi8_mask(input, CR);
All in all, speed-of-light for this code should be 3*3+2+11+1=23 cycles per iteration (if I'm counting everything properly), and probably even faster from pipelining between iterations. I hope it should be a bit faster than the 32ish that the old version was using.