Vectorscan – portable fork of hyperscan – high-performance regex matching lib | Hacker News Reader

Vectorscan – portable fork of hyperscan – high-performance regex matching lib

github.com

87 pointspabs39 comments

> Vectorscan is typically used in a DPI library stack, just like Hyperscan.

> Hyperscan is typically used in a DPI library stack. [https://www.hyperscan.io/about/]

Can anybody tell me what is actually DPI?

I only found the term "Data Processing Interface" and this paper http://cidrdb.org/cidr2019/papers/p11-alonso-cidr19.pdf

But this looks very theoretic, compared to MPI (message passing interface), which actually has a reference implementation and thelike.

DPI is deep packet inspection in this context. HyperScan did some groundbreaking work for DPI, where you (1) have to match a huge number of regexes at the same time, (2) need high and consistent performance, and (3) don't need the starting points of matches.

Groundbreaking in a cybersecurity context when you're writing regexes to match bad things in a fat network pipe.

Thanks, will switch over to this with my perl bindings (replacing the original perl core regex with vectorscan)

Congrats Intel, failure to play nice with community granted you a more up to date community fork.

Now let's play the internal discussion at Intel:

Developer: see !! I told you so, we had to integrate the patch. Community leadership is more important than these corporate plays.

Manager: err... Uhhh.... Now what do we do? How to regain community drive?

Developer: i don't know now.... Invest tons of time/money into hyperscan, have new chips that tap hyperscan natively, or re merge the fork?

...

“Wait, so you’re telling me we could sell a chip that only we could use to tap into for specialised performance?”

Is how I imagine that conversation going, but maybe I’m just cynical.

Cost of new die > market size, probably

NPU was exactly that...

And given scarcity, it's probable that foundries prioritize higher paying processors.

Just look at what happened to AVX512.

> Why the fork? Originally, the ARM porting was supposed to be merged into Intel's own Hyperscan, and 2 Pull Requests had been made to the project for this reason (1, 2). Unfortunately, the PRs were rejected for now and the forseeable future, thus we have created Vectorscan for our own multi-architectural and opensource collaborative needs.

Karma's a bitch.

Reply on news.ycombinator.com