Hacker News Reader

yeah there has definitely been work done in that space: it’s called multi-modal models

not sure if this is the latest work but here’s some results from Google’s AI Blog

https://ai.googleblog.com/2017/06/multimodel-multi-task-mach...

What would be really cool is neural networks with routing. Like circuit switching or packet switching. No idea how you would train such a beast though.

Like imagine the vision part making a phonecall to the natural language part to ask it for help with something.

Sounds like The Society of Mind - https://en.m.wikipedia.org/wiki/Society_of_Mind

Capsule networks have a routing algorithm as far as I know

Reply on news.ycombinator.com