Comment by saynay | Hacker News Reader

Comment by saynay | Hacker News Reader

Parent Full thread

One of the more interesting things I have seen recently is the combination of different domains in models / datasets. The top network of Stable Diffusion combines text-based descriptions with image-based descriptions, where the model learns to represent either text or images in the same embedding; a picture, or a caption for that picture, lead to similar embeddings.

Effectively, this can broaden the context the network can learn. There are relationships that are readily apparent to something that learned images that might not be apparent to something trained only on text, or vis-versa.

It will be interesting to see where that goes. Will it be possible to make a singular multi-domain encoder, that can take a wide range of inputs and create an embedding (an "mental model" of the input), and have this one model be usable as the input for a wide variety of tasks? Can something trained on multi-domains learn new concepts faster than a network that is single-domain?

I would love to see a model trained on blueprints or a model trained on circuit diagrams.

text2blueprint or wav2schematic could produce some interesting things.

They haven't even figured out basic math, so not sure what you would expect to find there. They aren't smart enough to generate structure that doesn't already exist.

Depends on the method. Evolutionary methods can absolutely find structure that we missed, and they often go hand in hand with learning. Like AlphaGo move 37.

AlphaGo had a lot of driver code involved to make it tick, it wasn't just a big network deciding what to do. You would need something similar here, without someone figuring out that driver code you aren't revolutionizing anything with todays neural networks.

AlphaGo without the search algorithm, just the neural net, operates at a professional (but not superhuman) level.

Later refinements with AlphaZero and MuZero also reduced the driver size.

The most naive form is just Monte Carlo tree search using the neural net as the bias/eval function. A basic MCTS is just 100 lines of code or so.

Yes, since Go is a very simple game. Making a proper driver for much more complex domains like engineering blueprints is not something we know how to do today.

Edit: Also you are missing the Go engine in that comment, it can't train without a Go engine to train against that evaluates the results of each move. That Go engine is a part of the training algorithm and thus is also a part of the driver code, you would need to produce something similar to train a similar AI for other domains. We don't know how to write similar blueprint engines or text evaluation engines, so we can't expect such AI models to produce similar results.

GPT3 can do some basic arithmetic.

It makes errors doing basic arithmetic's, so it just memorized some symbolic transitions and didn't figure out the underlying math.

Agreed, it's just rote learning.

Reply on news.ycombinator.com