Hacker News Reader

Comment by yarg | Hacker News Reader

> The author restricts general to mean restricted to text input and output. Which is obviously ridiculous.

That's not entirely true (but there should be a degree of plug and play adaptability).

Look, for example, at the very limited sensory domain of animals - you wouldn't deny their intelligence just because there's an information form that they are incapable of interpreting; and the information that they are capable of interpreting is that which they've been exposed to via their ancestoral heritage.

However, just because an AI doesn't understand a given input form, does not mean that it could not given adequate exposure; but it will need time to adapt and the closer the new information form is to one that it already understands, the faster it will incorporate the new understandings.

View on HN

>However, just because an AI doesn't understand a given input form, does not mean that it could not given adequate exposure; but it will need time to adapt and the closer the new information form is to one that it already understands, the faster it will incorporate the new understandings.

That is just completely wrong. There is absolutely no reason to believe that some language model will produce the same (or even similar) accurracy and training speed, if trained with image data as additional input. What you are claiming here is that any (sufficiently complex) neural net can be continually fed with different problem data and will output sufficiently accurate results. There is no reason why this should be the case and for the millions of nets which have been created none of them have exhibited this property.

It might very easily be the case that training GPT-3 with images means millions of times slower learning speeds for the same combined accuracy.

> That is just completely wrong. There is absolutely no reason to believe ... yada yada ... none of them have exhibited this property.

Go read up on pre-trained networks.

Are you familiar with the Gato paper from DeepMind this May? I say this not as a gotcha, but as a genuine question that I'm interested in your response to.

> Inspired by progress in large-scale language modelling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens.

> During the training phase of Gato, data from different tasks and modalities are serialised into a flat sequence of tokens, batched, and processed by a transformer neural network similar to a large language model. The loss is masked so that Gato only predicts action and text targets.

https://www.deepmind.com/blog/a-generalist-agent

An animal has a ton of data input.

Eyes, ears, nerves across your skin, group behavior, parents etc.

This to comprehend (alone the visual) is gigantic.

A brain also has an architecture which is NOT similar to what gpt-3 does

Alone the difference in how brains train alone is huge.

> An animal has a ton of data input.

> Eyes, ears, nerves across your skin ...

We have a limited but adequate amount of input, across very few domains. (I can't even see backwards.)

> ... group behavior, parents etc.

That's not sensory input?

I would argue it is. You need to see your parent and feel it and it's not random noise it's a grouping of different inputs.

Something which got doesn't has.

But you can give that another name if you like.

Reply on news.ycombinator.com