You're definitely right that we've found joint training to be highly useful and having a neural net simply act as a feature extractor is unsatisfying, but I think that success at unifying neural nets with symbolic or even non-learning systems would be a big step forward. This isn't it, but even the alphago work relied on Monte Carlo tree search, so I wouldn't dismiss this out of hand.