Lark-cython lets you double the speed of Lark's LALR parser (Python)

github.com

3 pointserezsh2 comments

I'm surprised the speedup for Lark is so small (<2X). Back in the Python 2.x days I wrote an LR(1) parser generator, and ended up porting it to Cython because it was taking several minutes to generate the parser I was working on. IIRC generation times dropped from over 3 minutes to less than 20 seconds, so ~10X speedup.

So, I think the main reasons are -

- Lark is already very fast. It's a lot easier to optimize something that takes 3 minutes :)

- There is still some Python code that is being run. For example, in the tree constructor. That can be solved in future versions.

- I'm not yet taking full advantage of Cython. For example, I'm still keeping the lookup map as a dictionary instead of an array. Also I'm still mostly working with CPython objects instead of native objects (like char*).

Reply on news.ycombinator.com