Hacker News Reader

Comment by gryn | Hacker News Reader

most of these knowledge graph papers seems to be about linking shallow content, likes citations etc,

with all these strides in AI, GPT, etc, does anyone know of papers about extracting into graph or other form of knowledge representations deeper semantics ?

for example instead extracting what the paper is about, how it relates to other papers, nodes and edges linking to field specific terms/concepts mentioned, translation of paper specific terminology to commonly agreed terminology, why it's citing said references (is it an example of use of a technique ? a proof for a claim ? something that's being refuted, etc ..), where in the text it's being used. and other kinds of "deeper meaning".

I've glanced at some papers that seemed to do something remotely close to this but it was restricted to doing NER for only medical/biology terms.

View on HN

The second layer is hard. I tried something in this space in mid-2018. Full text extraction and sentence segmentation tech was adequate, but extracting the discourse tree and building the graph was a bit of a struggle (trying to repurpose a collection of academic/open tools to get something useful). Never published or released the code.

If interested, a few rabbit holes to explore (no affiliations):

https://scite.ai -> best option for citation mapping, but same issues you described above

https://www.semanticscholar.org and AI2 -> the best group working on tooling in this space

https://www.weave.bio -> early startup trying to build this out

The hardest challenge in my view is solving the intermediate representation issue. You have to establish a DSL/nomenclature that provides the range required to represent a complete scholastic discourse while also being computable.

> The hardest challenge in my view is solving the intermediate representation issue.

Right, you'd basically be writing an interpreter for English

Have GPT translate down into a Controlled Natural Language. I tried having it translate to OWL, but it sucked.

https://en.wikipedia.org/wiki/Controlled_natural_language

I'm building this, for (mostly) non-scientific non-fiction works (books, articles, news, etc.). Launching soon, with about 7,500 books indexed.

Generally, what I found useful to build a graph between "topics" or entities was to use a HyDE[1] prompt to generate possible distinct definitions and then build a nearest-neighbor network from that. This successfully identifies related concepts in the truly abstract sense rather than literal entities.

[1] https://summarity.com/hyde

I'd love to chat! I am doing a rough version of this at Shepherd.com and laying the ground work for more :) (ben@re-moveshepherd.com)

I haven't uses Wikidata info yet, but hoping to expand to that in 3 or 4 months.

Sure. Sent a ping.

Hey @minxomat,

HyDE sounds like an interesting approach. All dense retrieval approaches suffer from the problem you outlined in the blogpost. Have you looked at keyword-based or late-interaction models for retrieval such as ColBERTv2[1]? I find that late-interaction methods seem to offer best trade-off between semantic intelligence (precision) and retriving relevant documents (recall).

[1] https://github.com/stanford-futuredata/ColBERT

Not a paper, but NLP extraction into knowledge graph representations do exist and are in use today. For example, here's a general purpose NLP model that links organization and people entities (among others) to each other based on factual relationships described in the text — https://demo.nl.diffbot.com

This semantic extraction can be extrapolated to most any trainable context. A useful one I've worked with involved mapping supplier-partner relationships. A well built supply chain graph can identify every layer of risk in a single supply chain and provide the provenance to back it up.

IMO, the biggest blocker to more mainstream use of Knowledge Graphs (even in the commercial world) is an actually intuitive interface for knowledge exploration. The real market innovation behind GPT isn't its 175 billion parameters, its the prompt interface that makes ChatGPT so universally accessible.

Lovely, for me coreference resolution also is a huge issue. I created this package (https://github.com/Pandora-Intelligence/crosslingual-corefer...) which is currently the only viable solution to do coref resolution in Dutch and some other low-resource language.

Also, if you want to scale, LLMs are going to prove to expensive, so eventually data need to be logged somewhere to create a fine-tuned model that can do sub-tasks and ideally do them better. What do you think?

Nice! Yes, coreference resolution is surprisingly absent even in enterprise NLP.

Depends on what you mean by "better". With more accuracy?

I think this is the next step in AGI - feed and reference general knowledge graphs from/to LLM. Then link to expert systems such as SAT/facial recognition/physics model/medical model.

I think LLMs will be capable of orchestrating between systems and maintaining a conscious narrative but knowledge graphs will solve the update and references problem.

You can mark reasons for paper citations using CiTO, the citation types ontology: https://sparontologies.github.io/cito/current/cito.html Not sure if there's anything standard along the same lines for more general annotation of e.g. Web-derived resources.

not a paper but a project: http://libscie.org

Reply on news.ycombinator.com