I built tilth (https://github.com/jahala/tilth) much for this reason. Couldn't bother with RAG, but the agents kept using too many tokens - and too many turns - for finding what it needed. So I combined ripgrep and tree-sitter and some fiddly bits, and now agents find things faster and with ~40% less token use (benchmarked).
There a lot of methods in IR/RAG that maintain structure as metadata used in a hybrid fusion to augment search. Graph databases is an extreme form but some RAG pipelines pull out and embed the metadata with the chunk together. In the specific case of code, other layered approaches like ColGrep (late interaction) show promise.... the point is most search most of the time will benefit from a combination approach not a silver bullet
Just like the approach in the article.
Everything is based on the metadata stored with chunks, just allowing the agent to navigate that metadata through ls, cd, find and grep.
> Switched to just letting the agent browse the directory tree and read files on demand -- it figured out the module structure in about 30 seconds
You guess what's the difference between code and loosely structured text...