herrington_d

How does it compare to teable[1], which is a postgres + airtable fusion?

I can see that Google Sheet has higher availability but fewer builtin dashboards.

[1] https://teable.io/

Is it possible to combine n-gram and AST to dump a better indexing?

Take `sourceCode.toString()` as an example, the AST can dump it to `sourceCode` and `toString`. A further indexer can break `sourceCode` to `source` and `code`.

For ast dumping, project like https://github.com/ast-grep/ast-grep can help.

Hi! This is Grit's release news so please allow me to only briefly answer your question. > nested syntax sg does not support it natively now and you need API to extract relevant text out.

For update code, I recommend first read the quick start, pattern and rule essentials

https://ast-grep.github.io/guide/quick-start.html

https://ast-grep.github.io/guide/pattern-syntax.html

https://ast-grep.github.io/guide/rule-config.html

Congrats from ast-grep team! Nice job Morgante!

GritQL shares a lot of common ground with ast-grep, which is also a code rewriter based on tree-sitter.

Interested folks may wonder about their difference. The key difference is that their surface APIs are quite different. GritQL is more like a DSL (or SQL). while ast-grep is more like pattern language + embedded configuration in YAML. Check it out here if you are interested https://github.com/ast-grep/ast-grep

thanks for mentioning ast-grep!

Thank for your suggestion! I will look at how to improve the README file!

> changing variadic C++ macros into arithmetic expressions

I don't know your exact use case but it sounds a little bit hard to me. I guess you want to change things like this.

```cpp

add_macro(a, b, c, d); // before

a + b + c + d; // after

```

It isn't that easy to do in pattern syntax. The pattern/replacement must support repetition and it isn't straightforward as far as I can tell from Rust's macro[0].

That said, if you need to support use cases like this, ast-grep has Python API to support programmatic usage[1].

[0] https://veykril.github.io/tlborm/decl-macros/macros-methodic...

[1] https://ast-grep.github.io/guide/api-usage/py-api.html

Hi! Thanks for trying out ast-grep.

I have been improving ast-grep's documentation since the release of the project.

It is arguably not that good/abundant for resources compared to eslint/babel. But it has improved a lot (say, example page[0] and deep dive page[1]). The doc site is also carefully crafted to make it accessible, compared to libcst or other similar projects.

I also appreciate blogs/introductions to ast-grep if the community can help! Let me know if there are issues/problems when using ast-grep!

[0] https://ast-grep.github.io/catalog/

[1] https://ast-grep.github.io/advanced/core-concepts.html

Hi, thanks for giving ast-grep a try.

Pattern code must be valid code that tree-sitter can parse.[0]

You can use this syntax `try { $$$CODE } catch ($$$E) { $$$HANDLE }`[1]

[0] https://ast-grep.github.io/guide/pattern-syntax.html#pattern...

[1] https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y...

Yes you can!

https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZ...

I confess I stole the pip recipe from Charlie :D

https://github.com/astral-sh/ruff/blob/main/.github/workflow...

Of course, it gets you covered.

https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZ...

I have the same problem also, haha, https://x.com/hd_nvim/status/1667059966111547392

Provable correctness means you have to model your source and target languages. And then translate the source model to the target model. It is theoretically possible, but in practice, modeling an industry language is way too much work. Some languages do not even have a spec :/

Why I think semgrep is a security tool different from ast-grep:

* Semgrep is security focused. It has many advanced static analysis features in its core product, such as dataflow analysis, symbolic propagation, and semantic equivalence, all of which are useful for security analysis. They are not available in ast-grep. * Semgrep’s pattern syntax also prefers matching more potentially vulnerable semantics than matching precise syntax. Semantic level information is the better level of abstraction for security model. ast-grep, on the other hand, sticks to faithfully translating users' queries syntactically. * Semgrep has a one-off search and rewrite feature, but it is not its primary focus. The CLI is a bit slow compared to other tools. ast-grep strives to be a fast CLI tool. * Semgrep has a product matrix for vulnerability detection: detecting secrets, supply chain vulnerabilities, and cross-file detection. It also has a plethora of security rules in the registry. These features will not be included in ast-grep.

Hi, ast-grep author here. This is a great question and I asked this in the first place before I started the hobby project.

TLDR; I designed ast-grep to be on different tracks than semgrep.

Semgrep is for security and ast-grep is for development.

First and foremost, I have always been in awe of semgrep. Semgrep's documentation, product sites and Padioleau's podcast all gave me a lot of inspiration. Using code to find code is such a cool idea that I never need to craft an intricate regex or write a lengthy AST program. sgrep and patch from https://github.com/facebookarchive/pfff/wiki/Sgrep have helped me a lot in real large codebases.

When I used semgrep as a software engineer, instead of a security researcher, I found semgrep has not touched too much on routine development works. I can use `semgrep -e PATTERN` but the Python wrapper is not too fast compared to grep. While pattern is cool, it cannot precisely match some syntax nodes. (example, selecting generator expression in Semgrep is very hard). It also does not have API to find code programmatically.

I have also a short summary for tool comparison. https://ast-grep.github.io/advanced/tool-comparison.html

Thank ekidd for your kind words! ast-grep author here. This is a hobby project and mainly focuses on developers' daily job like search and linting. Appreciate you like it!

Semgrep's vulnerability scanning is much more advanced, mostly for enterprise security usage.

treesitter gives us a uniform interface to parse and manipulate code, which is awe-inspiring work. I wish tree-sitter could have more contributors to the core library. It still has a lot of improvement space.

Say, like performance. tree-sitter's initial parsing speed can be easily beaten by a carefully hand-crafted parser. Tree-sitter, written in C, has a similar JavaScript parsing speed as Babel, a JS-based parser. See the benchmark https://dev.to/herrington_darkholme/benchmark-typescript-par...

Hey morgante, nice to meet you again! Indeed YAML is a compromise between expressiveness and easy-learning. Grit did a great job for providing advanced code manipulation!

Hey paulg, ast-grep author here! This is something I also want to do in ast-grep! ast-grep prints the surrounding lines around matches but they are not aware of which function/scope the matches are in. May I ask how you do the scope detection in a general fashion? (say language agnostic) https://github.com/ast-grep/ast-grep/issues/155

Native parsers used in JavaScript are not always faster due to extra work across languages. Avoiding these overheads and using multi-core are crucial for performance.

Thanks for the suggestion! I will modify it!

The missing guide for tree-sitter

Thanks! tree-sitter is highly useful but hard to use directly. I hope this tool can help tree-sitter be more used!

One thing I really like Verse is it makes prolog/datalog cute.

Verse is really a disciplined cue-lang on steroid. https://cuelang.org/

Evan You says:

    Maybe in the long run we want the whole transform (and even the Vue compiler!) to be implemented in native Go/Rust so performance would no longer be a concern ;)

Future is now!