A Brief Introduction to Graphical Models and Bayesian Networks (1998)

Causality is a bit harder to integrate with current machine learning models as it's hard even with standard probabilistic graphical models. On the other hand, there has been a lot of work integrating deep neural networks with probabilistic models.

For example, the variational auto-encoders are a graphical model with Gaussian latent variables whose mean and variance are determined by (deep) neural networks [1]. There has been work exploring the neural network weights as latent variables themselves [2]. Finally, some new developments such as dropout can be interpreted as some form of deep Gaussian processes [3].

I believe there will be a lot further developments on this area in the near-future.

[1] http://arxiv.org/abs/1312.6114

[2] http://arxiv.org/abs/1505.05424

[3] http://arxiv.org/abs/1506.02142

It feels like those are on different ends of a spectrum. On one side the Judea Pearl style causality analysis relies on human interpretation because we must adjoint the graphical network with an "assumption graph" which allows for the identification of suitable ad hoc experiments. This pretty much requires a physical semantics be applied to the PGM.

On the other hand, neural nets tend to eschew human interpretation. They optimize their objectives, but the resulting networks are rarely amenable to human interpretation. To try to come in after the fact and seek out experimental opportunities seems certain to fail.

On the other hand, it'd be interesting to see if the human assumption graph could be used as a training constraint for the ANN. I have never heard of anyone trying that.