Comment by gaogao | Hacker News Reader

Comment by gaogao | Hacker News Reader

Parent Full thread

I think there's a lot of cases where you don't want to just RAG it. If you're going for tool assisted, it's pretty neat to have agent write out queries for what it needs against the knowledge graph. There was an article recently about how LLMs are bad at inferring B is A from A is B. You can also do more precise math against it, which is useful for questions even people need to reason out.

I need to dig into what they're doing here more with their approach, but I think using an LLM for both producing and consuming a knowledge graph is pretty nifty, which I wrote up about a year ago here, https://friend.computer/jekyll/update/2023/04/30/wikidata-ll... .

I will say figuring out how to actually add that conversation properly into a large knowledge graph is a bit tricky. ML does seem slightly better at producing an ontology than humans though (look how many times we've had to revise scientific names for creatures or book ordering)

Yes, but this doesn’t seem to be an actual knowledge graph which is part of the issue imho. If you look at the Microsoft knowledge graph paper linked in the repo it looks like they build out a real entity-relationship based knowledge graph rather then storing responses and surface form text directly.

I think it's relatively unlikely that having an agent write graph queries will outperform vector search against graph information outputted into text and then transformed into vectors.

The related issue that I think is being conflated in this thread is that even if your goal was to directly support graph queries, you could accomplish this with a vanilla database much easier than running a specialized graph db

Outperform in what way? There's some distinct things it already does better on like multi-hop and aggregate reasoning than a similarity context window dump. In general, tool-assisted, of which KG querying is one tool, does pretty good on the benchmarks and many of the LLM chats cutting over to it as the default.

> if your goal was to directly support graph queries, you could accomplish this with a vanilla database much easier than running a specialized graph db

Postgres and MySQL do have pretty reasonable graph query extensions/features. If by easier, you mean effort to get up a MVP, I'd agree, but I'm a bit more dubious on the scale up, as you'd probably get something like Facebook and Tao.

Reply on news.ycombinator.com