I think the problem is that you are also trying to represent the biases of directionality in the connection e.g.: [-1,0,1]. In the adjacency matrix, if you store the sign (direction) of the edge, then I think that would map fairly isomorphically.