Comment by IIAOPSW | Hacker News Reader

Comment by IIAOPSW | Hacker News Reader

Parent Full thread

No. But with the caveat that it can only truly grasp things that are self contained systems of text. Allow me to give examples:

Transit wayfinding: A train service is nothing more than a list of stations it goes to, a station is nothing more than a list of train services that stop there. Nothing about the physical nature of trains or commuting is needed to have a discussion about train lines or to answer questions like "how many transfers does it take from x to y?". You could never have seen a train in your life or even a map of a train, if you've studied the dual lists of stations and services there's nothing more to learn.

Chess (and other board games): Standard Chess notation is a list of sequential move made by each player in the format [piece name][x coordinate][y coordinate]. Eg RB6 represents moving the Rook to column B row 6. Chess can be understood entirely in terms of a game of passing a paper back and forth, appending a new token each time, and the rules of the game expressed entirely in terms of how the next token must match the previous part of the list. At no point is it needed to have actually seen the physical board based representation of the game.

The machine is grounded in a text based existence. Conversational and linguistic objects are its literal physical objects. Anything outside of that, well it can understand lions and "large" the same we understand atoms and "eigen-states".

Experience may ground language, but you are free to ground language in a different reality with different basic constituents and relations. If AGI were to emerge out of trading bots, they would have a language grounded in money and trades. If it emerged out of a bot made to play Diplomacy, it would be a consciousness with the body of a nation state and the atoms of its world would be bits of plastic on a map of Europe. Grounding in our particular reality isn't strictly necessary, but it is helpful if the goal is to make something adept at the modalities of our particular existence and conversational form.

The problem with this line if thinking is that human language is not some self-consistent logical system you can reason about independently of its physical origins the way chess or transit or game systems are.

Instead a large part of the meaning of language is carried by shared human understanding of physical objects and events.

Edit: to give an example, the phrase "is it warm outside?" can be plausibly replied to in several ways, but the meaningful ones require knowing something about location and weather and human biology. No pure language-based learner can give the correct response without some kind of sensory-based information (in practice, that would come from some "oracle" based on weather services and location detection in current systems).

Aside, but I'll meander to the heart of your point:

As an American who's lived in Canada and Europe, I'll freely acknowledge the superiority of the metric system to imperial measures. EXCEPT celsius.

With Fahrenheit, you can describe a temperature with a single digit: in the thirties, in the forties, etc.

In Celsius, you're forced to add an extra bit of precision: low tens, high tens, etc.

Celsius-lovers find this argument infuriating and will go through all sorts of contortions to justify it: But what about the boiling point of water?

Who cares about the boiling point of water? Scientists. Chefs. But 99% of the time we talk about temperature, we care about the weather.

And this is sort of the point we're both getting at. That human-like intelligence is of particular value to us, in addition to non-human superintelligence at particular domains.

postscript: The anti-Fahrenheit thing is more a discomfort and lack of familiarity. An easy technique for explaining the intuition behind fahrenheit, for describing the weather, is "what percent 'hot' is it"? 80F is 80% hot. Not totally hot. Good-natured people will usually appreciate this rubric. The sort of people who enjoying sorting grains of rice will persist in winnowing the chaff here and perhaps point out that I'm mixing metaphors.

The boiling point of water may not be very significant to daily human experience, but the freezing point is, at least for people who don't live in the tropics. Celcius does better there than Fahrenheit, doesn't it? What "percent cold" is it in Fahrenheit land?

Also I don't like your argument that "only scientists" care about the boiling point of water because we live in a scientific society and for it to function well it is extremely important that everybody has at least some basic scientific literacy. Having separate temperature scales for scientists and "common folk" is at least one little step away from that goal.

> Having separate temperature scales for scientists and "common folk" is at least one little step away from that goal.

so you'd actually prefer to get weather forecasts in Kelvin?

Biologists and chemists, and yes, of course, meteorologists and climatologists, routinely use Celcius, not Kelvin, in scientific publications. So that's a strawman.

The point is that scientists use units appropriate to their domain and community. The general public uses units appropriate for their uses. These usually do not coincide.

There would be advantages (e.g. for standardization, "science literacy") if they used the same units but it would also be impractical for whichever side adjusts to use the other's units. I believe the advantages of such a switch are very small and the downsides small. Neither scientists nor the general public seem to think this is a problem.

Sure there are terminal words at the bottom. Natural language sentences eventually come down to relating some raw physical objects to others. We live in a reality where objects have inherent properties like size and texture and warmth and location. That's not a priori better or worse than the reality where objects have inherent properties like being a station or a service.

>No pure language-based learner can give the correct response without some kind of sensory-based information

No pure classical reality learner could ever come up with quantum mechanics. They must have a sensory organ to feel the eigen states.

> No pure classical reality learner could ever come up with quantum mechanics. They must have a sensory organ to feel the eigen states.

That's obviously not true, as biological organisms who lack such organs have still arrived at learning quantum mechanics (granted, we spent a few billion years inventing language first, but still, those tiny little microbes evtually learned how quantum mechanics works).

I should also mention that we don't yet know whether the eigenstates exist in any sense, or are just a cool modeling trick to predict probabilities, or are actually wrong and there is some better way to model particle physics that perhaps doesn't require them at all.

>That's obviously not true

Yes, that was my point

> Natural language sentences eventually come down to relating some raw physical objects to others.

Counterexample: Green is the color of hope.

That still ultimately refers to physical objects like human minds and printed symbols.

If you want a purely abstract statement, "the square root of four is two" does better.

Squares are physical. Historically exponents were only understood as measurements of the diagonal on a square, a neat trick for architects building something in our physical reality.

I acknowledge your point that there might be rich ineffable depths that could be present in the game of Go that are beyond the ken of human understanding or even imagination.

But I fail to see how an AI whose entire domain of knowledge is limited to board games, or financial trading, would qualify as artificial general intelligence. Unless you take a radical decentering viewpoint that human experience is just one kind of intelligence, and aren't all kinds of intelligence equally valid?

That would be glib. Of course we want superhuman drug discovery algorithms that don't understand the pleasure of a sunny day. But most people, when they speak of the topic of AGI, mean: encompassing the scope of human experience.

>But I fail to see how an AI whose entire domain of knowledge is limited to board games, or financial trading, would qualify as artificial general intelligence.

Let me recast this:

>I fail to see how an I whose entire domain of knowledge is limited to low energy physics, or the large mass limit, would ever qualify as general intelligence.

You see the problem. You can extrapolate just fine beyond the reality of Newtonian physics and Euclidean geometry which is native to you. A trading bot AGI could talk about things that aren't money, it would just be using money related objects with money related properties to tell this narrative picture of a world which isn't money. Not too different from the sort of analogies we use all the time to map hard concepts back to the space we find natural.

So let me answer your question with a question. What makes a substrate of properties like "size" and "color" and "texture" a better reality to be grounded in than properties like "buy price", "sell price", "dividend"? Is the former uniquely suited for intelligence to generalize beyond that given domain, or is it possible that all groundings are good as any other?

Very interesting question! I would argue that concepts like "size", "colour", "texture" are grounded in a more base level of reality than concepts such as "buy price", "sell price", "dividend" etc. The former set of concepts relate to aspects of the physical world as perceived by creatures that inhabit that world, while the latter refer to social constructs of the human socio-economic world that are themselves reliant on abstract human concepts of utility, value, the market etc.

This thought is not fully developed but I'm drawn to the idea that if an intelligence is grounded in an understanding of a more base level of reality it will find it easier to generalize beyond any one given domain. I could be entirely wrong of course.

But how do you determine the basicness level of reality? And how do you know there isn't a layer lower than our own?

I'm philosophically comfortable with the notion that some things are unknowable and that eventually one just has to take certain cornerstone propositions about the nature of reality on trust and / or faith. Ultimately though the question is irrelevant to the discussion at hand, as the claim I'm making is that physical reality is at a lower level than socio economic reality (which I'll give you is a contested claim these days)

Yes, the systems we construct within this reality are obviously not the bottom of the stack because we built the stack. But suppose the trading bots figure out how to manipulate the rounding mechanics of high frequency trading to jerry rig a Turing machine within their reality. Now fast forward from this and suppose they've all obtained an equivalent of a home PC. For fun they all start playing a "sim city" like game version of our reality.

So far as they can tell, money is real reality (rather than socio-economic construct), and the "physical" reality is an abstract game constructed at a higher level.

So who's reality is who's construct in the end?

I get your point, but you're making my head spin here from the sea-sickness involved in being asked to acknowledge that all viewpoints are equally valid.

People care about human-like intelligence. That's the point. We also care about trading and physics. But when we talk about the specific question of human-like intelligence (and not other kinds of intelligence), the human reality is the appropriate substrate. Acknowledging, of course, that for non-human intelligence there might be other substrates and also acknowledging that non-human intelligence can be valuable to people.

But let me take your argument to the extreme.

In classical machine learning theory, there's a proof of the value of bias. Bias plays a crucial role in guiding the learning process. If all hypotheses are considered equally possible, learning becomes infeasible due to the lack of any preference or constraint on the hypothesis space. Bias, in this context, refers to the inherent assumptions a learning algorithm makes about the data's underlying structure. Introducing some bias into the model allows it to favor certain types of solutions, thus narrowing down the hypothesis space and making learning feasible.

The logical extreme of your argument---if I understand it correctly---is that all machine learners are equally valuable. For example, an AI that learns a domain that is completely removed from all human values and thus we would be completely agnostic to and ignorant of, ipso facto, because it does not pertain or relate to us at all. In which case, who cares? By "who", I mean people. No one. By construction.

[edit: I'll give an example here. Let's say I randomly pick a corpus of images that are pure noise. I induce an algorithm to model that noise. That model will learn something valid but in a domain completely divorced from anything with human impact.]

So if we can acknowledge that some forms of learning are more important, merely by virtue of the fact that humans have preference, then perhaps we can perhaps wend our way back to agreeing that human-like intelligence is one valuable kind of intelligence, in addition to other valuable forms of intelligence like trading or board games.

We can play the game all day of "everything is subjective" and "up is down and black is white", but at the end of the day, for human beings, it's night.

The "general" in "general intelligence" is when you can hack together a picture of other domains by stitching together objects in your own domain. Our reasoning system only evolved to talk about 3 physical space dimensions and 1 time dimension and Newtonian physics and euclidean geometry. Things like trades and ownership and train lines aren't basic objects in our domain and the abstract rules governing their relations are completely unlike those rules of object properties like heavy and bright and loud. Can an intelligence stuck in a different substrate also come up with abstractions and generalize its intelligence? Maybe

>The logical extreme of your argument---if I understand it correctly---is that all machine learners are equally valuable.

That's not quite it. My argument is that general intelligence can in principle be grounded in the semantics of pretty much any baseline of objects and relations. The specific base line properties such as size, shape, color etc is orthogonal to the property of intelligence. We don't need to give the machine familiar sensory input for general intelligence, and general intelligence would not imply that it can do well at conversation in our domain (merely that it can talk about our domain in a very contorted way if it had to).

I'll admit if the goal is to make an intelligence "in our image" then sensory modalities will get us there faster. It will also lead people to erroneously believe that the key ingredient to general intelligence is physical grounding in our reality. My counter example to that belief is general intelligence embedded in a pure language model, which is perfectly recognizable as "in our image" AGI just so long as you stick to topics like transit lines and chess and whatever else can be grounded in words alone.

The definition of „general intelligence“ as chess, transit routing, and pure language, is not a commonplace interpretation of this term of art. But with that interpretation in mind, I see your point.

It can still be "general" outside of those areas, it will just be prone to mistakes of common sense. So to rephrase it all tersely:

Sensory grounding isn't needed for general intelligence or for computers to be capable of "understanding" (as per the title), but it is needed for that intelligence to have what we consider common sense.

But I'm glad I got my point made. Great talk.

Reply on news.ycombinator.com