Hacker News Reader

Comment by cedws | Hacker News Reader

Going to hijack this to ask a question that’s been on my mind: does anybody know why there’s no agentic tools that use tree-sitter to navigate code? It seems like it would be much more powerful than having the LLM grepping for strings or rewriting entire files to change one line.

View on HN

There are tools for that, such as:

https://github.com/wrale/mcp-server-tree-sitter

https://github.com/nendotools/tree-sitter-mcp

https://github.com/NightTrek/treesitter-mcp

These links are all mostly unmaintained AI-generated random MCP servers. Could have put in a bit more effort than copy-pasting search results...

To talk about where it's _actually_ at:

Agentic IDEs have LSP support built in, which is better that just tree-sitter -such as Copilot in VSCode, which contrary to what you might expect, can actually use arbitrary models & has BYOK.

There's also OpenCode from the CLI side etc. etc.

From the MCP side, there are large community efforts such as https://github.com/oraios/serena

In my own measurements (made a framework to test number of tokens used / amount of reprompting to accomplish a battery of tests) i found that using an ast type tool makes the results worse. I suspect it just fills the context with distractors. LLMs know how to search effectively so it’s better to let them do that, as far as I can tell. At this point I basically dont use MCPs. Instead I just tell it that certain tools are available to it if it wants to use them.

This doesn't fully satisfy your question, but it comes straight from bcherny (claude code dev):

> Claude Code doesn't use RAG currently. In our testing we found that agentic search out-performed RAG for the kinds of things people use Code for.

source thread: https://news.ycombinator.com/item?id=43163011#43164253

Thanks, I was also wondering why they don't use RAG.

I forgot to mention that Aider uses a tree sitter approach. It's blazing fast and great for quick changes. It's not the greatest agent for doing heavy edits but I don't it has to do with them not using grep.

They are using RAG. Grep is also just RAG. The better question is why they don’t use a vector database and honestly the answer is that these things are incredibly hard to keep in sync. And if they are not perfectly in sync, the utility drops quickly.

By RAG they mean vector search. They're calling grep "agentic search".

Create an agent with these tools and you will. Agent tools are just functions, but think of them like skills. The more knowledge and skills your agents (plural, Id recommend more than one) have access to, the more useful they become.

This is how the embeddings generation works, they just convert it to embeddings so it can use natrual language

> does anybody know why there’s no agentic tools that use tree-sitter to navigate code?

I would guess the biggest reason is that there is no RL happening on the base models with tree-sitter as tool. But there is a lot of RL with bash and so it knows how to grep. I did experiment with giving tree sitter and ast-grep to agents and my experience the results are mixed.

Reply on news.ycombinator.com