I have thought of the same thing in the domain of neural nets. You want to understand an image? Build a graph of the objects and their relations. You want to understand a text? Turn it into a graph. Need to simulate the environment? You can represent it as a graph. Want to mine a huge database of common facts? Represent it as a graph (database of triplets subject-relation-object). Even the execution of a program could be better conceptualised as a graph.
Graphs seem like an universal format. Graph Neural Nets have been a blooming subfield lately, ever since Thomas Kipf's paper [1] which in three years has accumulated over 2300 citations.
The main difference between classical neural nets and GNNs is that GNNs have permutation invariance. You can rename the nodes of the graph and still have the same graph.
Previously neural nets only had translation invariance (CNN) and time invariance (RNNs). Graphs would solve the combinatorial explosion by virtue of their permutation invariance - a problem which limits previous models of neural nets. Instead of learning all possible combinations of input objects you learn the pairwise relations, then you can generalise those relations to new configurations.
On a side note, the insanely popular transformer architecture (based on soft attention) is a kind of implicit graph, where the connectivity is evaluated based on the dot product similarity of the key and query objects.
[1] https://scholar.google.ro/scholar?hl=en&as_sdt=0%2C5&q=kipf+...