Deep Graph Library: Easy Deep Learning on Graphs | Hacker News Reader

Deep Graph Library: Easy Deep Learning on Graphs

dgl.ai

122 pointsbased218 comments

There is also a GeometricFlux.jl[1] for Julia language and FluxML[2] framework.

[1] https://github.com/yuehhua/GeometricFlux.jl

Anyone know how this compares to Facebook's recentish work on PyTorch-BigGraph (https://ai.facebook.com/blog/open-sourcing-pytorch-biggraph-...). They claim to be able to embed billions of nodes. I presume this library isn't distributed (quick glance suggests not).

They serve different purposes. PyTorch-BigGraph (BG) implements a set of algorithms for learning node embeddings (vector representations of each node in the graph) based on the edges (relations) present on a single graph.

dgl is a library for graph neural networks (GNNs). The algorithms present in BG can be implemented in dgl, albeit much less efficiently but the reverse might not be true.

More specifically GNNs are a set of methods based on what is called "Message Passing" algorithm, where the embedding of each node is a function (parameterized over the model weights) of its neighborhood and the edges that connect the node to it.

Additionally GNNs target learning functions that work on multiple graphs for example graphs of molecules to predict their properties, not just a single graph.

dgl can be compared to PyTorch Geometric. The former works on both TF 2.0 and Pytorch while the latter is only for PyTorch.

Both are almost equivalent, although dgl has some institutions backing it. PyTorch Geometric might feel a bit more lightweight to integrate in existing codebases.

Check this out: https://www.amazon.science/publications/distdgl-distributed-...

Feeling dumb:

Does this thing learn a graph (i.e. build nodes and edges) from more unstructured data like text in order to make relationships explicit?

Does it take a knowledge graph and use it as partial input to help learn some other function?

Or something else?

The latter. It does not infer graphs from unstructured data, it gives a set of backend agnostic APIs for constructing featurized graphs, message passing and pooling/aggregating (with gradients being handled by your framework of choice), and has some really nice helpers like performing softmax over certain node/edge types, while keeping everything performant.

For a Tensorflow/Keras alternative check out Spektral: https://graphneural.network

What do people use Graph Neural Networks for?

It's a generalization of a fully connected layer, where you give the network an adjacency matrix for data routing between nodes. This allows for designing sparse data flows and learning relations between pairs of input elements.

Graph NN layer: f(X, A) = g(AXW), where g is an activation

Fully connected layer: f(X) = g(XW), like GNN where the adj. matrix A=I

To put it in context a few years ago GNNs became a hot field. They are very similar in a way to transformers because both do pairwise interactions between elements, the difference being that GNNs use explicit and transformers implicit graphs.

Is this a simplified interpretation of GNNs? because I couldn't find a reference for that.

EDIT: found it! https://arxiv.org/pdf/1609.02907.pdf

A routine automation task is information extraction from any kind of document with a visual layout - receipts, PDFs, web pages. [1] [2]

[1] : https://nanonets.com/blog/information-extraction-graph-convo...

[2] : https://arxiv.org/pdf/1903.11279.pdf

Traffic prediction https://deepmind.com/blog/article/traffic-prediction-with-ad...

The transformer is a fully connected graph neural network. Read https://arxiv.org/abs/2004.08483 for further ideas.

Chemical structures can be described as graphs.

They’re seeing some interest in neuroscience as the brain can be represented with a graph structure.

fraud detection

expanding on that: people used to do stuff like pagerank etc. ahead of time and decorate their tabular data with the results: entities, events, whatever for social, fraud, security, customer journey, whatever. that meant a phase separation between graph analytic enrichments and learning. bringing the non-local graph reasoning to the learning phase in a way that isn't slow enables closing the loop.

The early graphsage stuff was, afaict, proven for generic social recommendors, but most gnn's I see seem pretty custom (e.g., deepmind's protein folding solution), esp. when not prohibitively slow. It sounds like more generic use is becoming practical w/ these libs, and esp. interesting to me, the latest NIPS had graph transformers papers, which brings another level of practically here. Not sure if DGL & friends have those yet..

Quite handy, also check out dgl-lifesci for utilities dealing with molecule-graph-specific GNNs

Reply on news.ycombinator.com