It would have predictible structure. There would be groupwise, regular sparsity. You could exploit these patterns to infer properties like, "there is no connection from this vertex to this entire span of verticies".

Just think about representing a compressed version of a matrix. You want to have a short encoding for matrices are ones that are more ordered in some real, exploitable way. With some determnistic computation of the compressed representation, you could produce the full adjacency matrix.

I think the problem is that you are also trying to represent the biases of directionality in the connection e.g.: [-1,0,1]. In the adjacency matrix, if you store the sign (direction) of the edge, then I think that would map fairly isomorphically.