This does not make any sense to me and neither did OP's comment about NN's approximating the posterior. In fact, if p were the solution then that would simply be the maximum likelihood estimate, which would not include the p(theta), or the prior, and hence would not be Bayesian.