Artificial Intelligence Hits the Barrier of Meaning | Hacker News Reader

Artificial Intelligence Hits the Barrier of Meaning

nytimes.com

213 pointsmathgenius148 comments

This latest craze of "AI" research seems to be fueled by a sudden glut of computational power (GPUs) that wasn't available previously. I think that most technical people would agree that the mid 2020s is extremely ambitious. I'd also argue that we're actually more likely to experience another AI winter.

The frightening part of the current deep learning research is how susceptible they are to adversarial attacks. Adding small amounts of noise causes misclassification in images, and some papers even explore the inevitability of adversarial examples [1]. This is especially frightening given the amount of autonomous vehicle work being done. I could imagine a situation in which the sensor noise varies just enough to cause such an error. Obviously, the systems will have redundancies built in, but I'm convinced the self-driving cars are still a ways off as well.

EDIT: As others, have stated just adding noise is not enough and it is often used to generalize the model. The paper does discuss that the perturbations can be incredibly small to cause this deviation and that the set of such deviations may be larger than expected especially for complex images.

Regarding the AI winter, I suppose I should have defined it as a reduction in the amount of research and the extent of the progress being made in the area rather than the utility of such research.

[1] https://arxiv.org/abs/1809.02104

This is a common misconception. It is not a small amount of noise that causes misclassification of images. It is a carefully designed and quite unique pattern that causes misclassification. It only looks like noise to the human eye, but it really isn't.

Yes, neural networks are susceptible to adversarial attacks. No, just adding noise to an image doesn't break neural networks.

Adding small amounts of noise is actually sometimes used to improve the performance of various AI techniques. It helps prevent overfitting.

In fact, if your technique or model is seriously affected by a little noise this is usually enough to brand it brittle and maybe even a failure, as it's a sign of overfitting. Anyone working in this field knows to look for this and will try to make what they create more robust.

The design of visual captchas is one obvious indication of just how successful AI techniques have been at image recognition in the presence of noise. It's no longer enough to make them a little noisy. In order to resist being solved by mechanical means, visual captchas have to include so much noise that even humans have problems recognizing them.

Read it as: A small amount of carefully constructed noise. Then you are correct to literature and pop-science. No misconception needed. There are 1-pixel attacks now. Randomly shuffling a small amount of pixels around can cause predictions to shift.

The issue is that there is no scene understanding. No common sense. No 3D modeling. Just 10x10 pattern matching on a very large fuzzy database of natural images (which works really really well in most cases).

The hype of ML is driven by 3 things: Big companies vying for AI dominance, militaries that want to finally use neural nets that work, and international competition between the West and the East to be the first to largely automate their economies (or AGI if you want to call it that). Catalysts were big data hoarding, GPU training on ImageNet, and then AlphaGo.

I am not a machine learning expert, but could not these adversarial example issues be resolved by solving an image classification problem by (1)producing multiple non-equivalent classification solutions with adequate accuracy, then (2)fusion (e.g. voting) to produce a consensus classification? (3) Maybe random shuffling of which X of Z solutions get to vote in each classification attempt.

What might fool one solution might not fool another, and adversarial examples seem to depend on idiosyncrasies of a particular solution.

Or just adding a small amount of random noise to the input, which would wipe out the carefully constructed attack.

You can try out this technique at https://github.com/google/unrestricted-adversarial-examples My guess is it would have the same result as adding noise to the normal images too (resulting in a slightly worse performance overall).

When we do that (and we usually do it for accuracy improvements, not resistance to atrack, that happens at the same time for the same reasons - adversarial examples are a misclassification problem), we change the exact attack that breaks the system, but there is no 100% accurate system - and since it's completely foreign, the examples that a machine would misclassify would likely not confuse a human.

The issue is classification currently relies on a very small embedding of the data which is pattern-matched, with no semantics. It has no way of telling that the difference between a dog and an elephant ISN'T that noise gradient, at least some of the time!

Some of them yeah. There is active research on this. But it is also possible to create adversarial images for soft voting ensembles of the 6 most popular architectures. Those strong adversarial images that beat the consensus, also have a large chance to fool new neural network architectures that the adversarial image creator never had access to.

Can I have a tool to add this noise please? So that Facebook et al. can't find me and build a profile on me based on random images that I didn't even know existed?

https://cvdazzle.com/

That is a really cool research/art project!

Nice. I just sent that to a friend who has a hair salon in SF.

This is the most cyberpunk thing I've ever seen

This style made an appearance on Elementary a few years ago: https://www.youtube.com/watch?v=A1_9aHo0S30

Sounds like those networks need an adversarial network or two to improve their performance and make them less susceptible to attack.

> I'd also argue that we're actually more likely to experience another AI winter.

We'll experience an AI winter again like we experienced an Internet winter in 2001-2004. Which is to say, not really at all. AI is now being widely commercialized for the benefit of consumers and businesses. That process will not stop, even if the hype train deflates before rising again at a later date. There is large, tangible commercial value in AI at the current general level of capability and near-term potential. That will result in pursuing maxing out whatever this era is capable of, before the next leap occurs at some point down the road. It's a progress track of higher highs during the exploratory boom and higher lows during the winter.

> This latest craze of "AI" research seems to be fueled by a sudden glut of computational power (GPUs) that wasn't available previously. I think that most technical people would agree that the mid 2020s is extremely ambitious. I'd also argue that we're actually more likely to experience another AI winter.

I think that is extremely unlikely. "AI" (read: machine learning) is actually being used for business purposes now, it's delivering enormous value to nearly every business on the planet. We're now in a long phase of descending the gradient of the current batch of broad techniques. This is likely a decently long gradient, with lots of marginal improvements to be made for a long time. And whereas with research projects, people don't care much about marginal improvements, they really do for business use-cases. For those reasons, I think AI/ML is basically here to stay just as much as basic biological research, or physics, or whatever is, if not more.

>> "AI" (read: machine learning) [is] delivering enormous value to nearly every business on the planet

That statement appears to contain two fairly bold claims - could you share sources?

Every big website uses some kind of machine learning to prevent fraud. Banks do the same. Data mining is used everywhere to improve customer experience. Data mining is used in industrial applications for preventative maintenance.

Not OP, but of course even 0.1% improvements are worth millions to search engines, social network feeds, financial forecasters, and self driving car companies. Also worth thousands or millions to manufacturing processes, and to small businesses, which might be using ecommerce optimization systems through providers.

I don't think adversarial examples give any evidence of relevant problems with these models because they occur on a very specific subset of images that can only be discovered using detailed knowledge of how these networks process images.

For all we know, humans have similar problems on some obscure subset of images, but we can't find human's adversarial examples because we don't have detailed knowledge of how the brain processes images.

I think adversarial examples for humans are called "optical illusions".

There is a categorical difference between "a [specifically designed] image that can be construed as a duck or a rabbit" and "a human can regularly mis-categorize random pictures of ducks as rabbits if a weird filter is overlayed". The first is well-known and fun and trite -- the second is unheard of and probably impossible for humans, yet provably possible for trained computers.

The point outlined is that we don't know enough about how we identify objects to discard a simple adversarial attack; probably not a filter-based but maybe something else.

It's called camoflauge. The natural world is full of adversarial examples.

"probably impossible for humans"

Based on what?

I'd imagine GP was referring to "humans perceive straight lines to be curved when certain shapes are overlayed", or "humans perceive shapes of the same color to be different colors when filters are applied" sorts of optical illusions.

There are plenty of those, and I personally I think they're probably analogous to how adversarial filters fool AI classifiers.

It's really easy to cause humans to misclassify all kinds of images as containing faces ;). Humans also regularly misclassify random noise as words. You can even suggest which words we hear by telling us what the noise is supposed to be.

The difference is that humans are aware that there is an illusion happening, they just can't help seeing it.

Or sometimes not aware

https://www.dw.com/en/man-falls-into-black-hole-art-exhibit-...

The important difference here is that most adversarial examples for the human perception:

(a) do not occur frequently in nature,

(b) are not frequently - if at all - produced in man-made architecture or transit-constructions,

(c) often contain repetitive and regular geometric and chromatic patterns which further make them stand out from everything else, and

(d) practically cannot be produced by digital (ergo noisy/less-than-perfect) images of any common real-world scenario.

In short: optical illusions don't accidentally occur in places where they can be seen by meatbag drivers.

I don’t see how you can make any claim about “most human adversarial examples”. There is a huge space of images and we have explored a negligible part of it.

Also a) and b) empirically seem to be true of the test sets people have collected thus far of the natural world for these models.

In short, we have no evidence that adversarial examples of the type being studied occur commonly in images collected by self driving cars.

The issue with regard to self-driving cars is that these cases demonstrate a disturbing level of fragility: we don't have a good handle on where the boundary between acceptable and chaotic responses lies.

You hypothesize that there are comparable examples for humans somewhere out there in the domain of all possible images, but the fact that, for all the countless cases of people looking at things that have occurred in humanity's existence, no-one has found a good example, suggests that, from the pragmatic point of view that you propose, image-recognition software has some catching-up to do.

Maybe a system that seeks consensus among several differently-trained models would be more robust.

The difference is that you can calculate an adversarial example for our classifiers, but it's too slow to calculate on a human.

Even if you could, the result would be specific to that particular person, so it won't work as good on others. And these bastards learn while you're constructing the example (which isn't fair at all to a helpless classifier that's just sitting there and doesn't change).

Fairness doesn't come into it - machine vision has to be up to the task it is given, period. If humans depend on their more general intelligence to deal with problem cases, machine vision either has to do something similar, or compensate adequately in some other way.

That was a joke.

https://arxiv.org/pdf/1802.08195.pdf

Looks like we are starting to find examples.

I think your intuition is wrong because humans are adapted to what exists naturally so of course there are no naturally occurring adversarial examples. It seems like the same is true for models trained on large natural image sets though.

My point is not wow let’s stop developing neural networks they are perfect. It’s more let’s go collect real world test sets to find and then fix gaps. Adversarial examples actually help very little in making nets more robust in the ways that matter.

> The important difference here is that most adversarial examples for the human perception:

> (a) do not occur frequently in nature

You've never heard of walking sticks? Ever seen one of those leaf moths?

If you had seen one, would you realize you had?

Is a deer visible on that stretch of hillside, or is that just dead grass?

Well yes adversarial images cannot appear naturally because by definition are created out of the network itself, but they highlight the same issue that caused misclassification of the lady in the Uber incident.

I think we do actually know enough about how the human mind does process images to have some idea of what is different. It is not that uncommon for humans to be uncertain about what they are looking at, but the first thing about such occurrences is that the human is usually aware of the fact that they are having a problem, and the second thing is that they take steps to resolve it, such as making hypotheses as to what's going on and checking them out, and/or seeking to get a better view (or other evidence) in a way that is specifically designed to resolve the uncertainty. It is this higher-level semantic analysis that is missing from current image processing software.

In these discussions, someone always mentions optical illusions, but only humans (so far) understand the concept of 'optical illusion', and recognize that they are experiencing them.

> It is not that uncommon for humans to be uncertain about what they are looking at, but the first thing about such occurrences is that the human is usually aware of the fact that they are having a problem, and the second thing is that they take steps to resolve it

This is true, but step one is "move your head" (or in your words, "get a better view" -- but you get more value from just the fact that your head is in a different place than from the possibility of a better angle on whatever you're looking at).

That strategy doesn't work at all when you're trying to classify static images rather than physical objects.

GP's statement applies as much to observing objects in 3D space as it does to looking at photos, where just moving your head ain't gonna help you much. Optical illusions are great to study this process, because most of them are delivered in form of flat images on paper or computer screen.

Optical illusions are delivered as flat images because moving your head doesn't affect those.

That raises the interesting question of how object recognition in streams of images is progressing, beyond being just object recognition within the individual frames. Humans are capable of extracting a lot of additional information in such situations, and are actually helped when the perspective on a given object changes. One cannot give current machine vision a pass if, through lacking this capability, it is under-performing.

And moving one's head to get a a better view is only one thing that a human might do. Firstly, of course, we must recognize that we are having a difficulty, and current machine vision seems to be somewhat deficient in this regard. Then, even without being able to get a different perspective, we will do things like make guesses as to what might be there (using our extensive semantic models of the world) and figure out if they might be a good fit to what we see, and/or we might try to extract specific features of the problematic area and search our memories for objects that might plausibly match, bearing in mind that it might be from a different perspective than we are accustomed to. We are also quite good at estimating whether an object might be a problem for us, even if we have not positively identified it. There is a lot more to it than just moving one's head.

Humans are rarely aware of optical illusions unless they're extreme images they don't see in real life - crawling dots, impossible geometry - or they're explicitly labelled as optical illusions.

Some more subtle examples:

http://www.terrycolon.com/1features/optical-illusions.html

In fact human perceptual processes are only kind of reliable some of the time. Low and/or unusual light, suggestibility, and unusual contexts all have a very negative effect on reliability, but humans are often unaware of this.

Cognitive and semantic illusions are even more persistent. People literally believe all kinds of nonsense, and will carry on believing it even when offered robust evidence that they're wrong.

The point being that human perception and cognition are not some kind of gold standard. They have plenty of issues of their own. But there's a kind of assumption/requirement of perfection with machine intelligence that doesn't apply to human cognition. So bugs in our own evolutionary firmware tend to be overlooked, while equivalent-level bugs in ML are seen as terrible failures which undermine the entire premise of AI.

Recent update — we do know, and humans are vulnerable to perturbations created to fool a multitude of existing AI: https://arxiv.org/pdf/1802.08195.pdf

Not true. There are black and grey box adversarial techniques as well.

I don't think it is just a matter of computational power: I have been quite surprised by how effective word embedding, for example, has been in abetting language translation (note that I am not claiming that it has solved the problem; I am too familiar with the problems of idiomatic Vietnamese-English translation.) Of course, if you had different intuitions (or more knowledge) than me, you might not be so impressed.

Having said that, I agree that the projections seem highly optimistic, but maybe I will be surprised again.

> how susceptible they are to adversarial attacks. Adding small amounts of noise causes misclassification in images, and some papers even explore the inevitability of adversarial examples

What if, in a distant future, computers turn out to be the correct one, humans's perception are biased?

In what sense an image of a rabbit perturbed by adversarial noise can be _correctly_ recognized as a duck?

There can be a general consensus that the image looks like a duck at most. And if humans see a rabbit and AIs see a duck there just won't be consensus.

>I'm convinced the self-driving cars are still a ways off as well.

Technology-wise, absolutely they are. The problem is that in actuality, they aren't. Companies will continue to push as hard as they can for as wide of a launch as they can, while governments (and any kind of sorely-needed oversight) will be ages behind.

Great article by Melanie Mitchell (she gave me her code for the Copy Cat program from her book with Hofstaeder almost 30 years ago - she has been looking at creativity and common sense in AI for a long while)

My day job is all about deep learning but personally I think we need to stop and take a deep breath and really solve problems with biased data sets and models, easily spoofed models, etc.

I have worked through a few AI winters and we may be hitting another one. I would like to see care given to using deep learning models only where it is safe to do so.

Yup. Dotcom era all over again where everyone and their grandmother need to be building a website.

Dotcom era gave us Amazon and Google. If AI craze gives us something like that again, I think that's fine.

There have been impressive advances in Computer Vision; they're just an outgrowth of the work that produced JPEG, rather than of the work that produced Lisp machines and expert systems.

It's often said that we tend to take AI that really works and call it something else, but we also tend to call whatever is current and somewhat succesful (fuzzy logic, etc.) "AI".

Universal surveillance and data collection?

/Cynic

I know I'm late to the party, but...

Amazon abuses many of their workers, they've been dangerous to small businesses in any area they move into, and they're contributing significantly to the wealth inequality problem.

Google knows a scary amount of information about you and everyone you know, they're supporting a horrific change in China allowing even tighter control of their citizens, and they were even looking at building AI for military drones at one point.

In exchange, we get to buy cheap stuff with quick shipping and we can find resources on the internet slightly quicker.

An AGI could have much more significant implications than either of those companies ever have. We need to figure out a solution to the AI control problem before we have a Dotcom-like burst of development.

I don't know much about AI, but I wonder what AI people think of this theory:

https://en.wikipedia.org/wiki/Embodied_cognition

The idea being that if we want to replicate strong AI, it needs to be embodied, because a lot of our cognition is built on metaphors that are instantiated in our physical actions and perceptions.

I believe this to be true (embodiment as a precursor to cognition). So much of our early years learning is exploring how our various actions lead to a corresponding result. Touching a hot stove vs petting a soft cat, for example. And instead of just training 1 neural system at a time we're training all of our senses at once, which I think gives a significant boost to the brain's 'understanding' of the result of the action.

Embodied cognition probably matters a lot if you're trying to build a drop-in replacement for humans. But in principle if someone manages to build a disembodied AGI it would probably still be useful for many purposes including ones we haven't even thought of yet. Think of it like a special-purpose dedicated co-processor. We already have those for other purposes like math, physics, graphics, etc to perform specialized tasks that the CPU can't do efficiently.

In practice we're nowhere close to being able to build any sort of AGI so this is just a thought exercise.

One (of many) reasons I agree is because determining causal relationships is far easier when you can perform 'experiments', i.e. make changes to the world see the results.

Another reason is that nearly all of what we call "common sense" is just knowledge about the real world rather than being some kind of abstract reasoning ability.

Note that embodied cognition doesn't require robotics. An agent can act in a simulated environment instead.

Note also that Deep Mind is very heavily focused on embodied agents.

I think we don't need to go there yet. There needs to be a stage before that which is the ability to precisely debug the human brain. If you don't understand the brain, then embodied cognition will be impossible to understand. I think the theory needs to be tested in much more depth. Fortunately, you could create AI models to test the theory itself. AI can serve as a testbed of our understanding in biology.

Let me dive in on the idea of debugging the brain.

If we're able to fully record one's brain activity in a precise manner, then we'll understand much better how to create an intelligent system.

This is because very strong advances have been made in machine vision through a similar idea. Scientists didn't need much precise granularity to understand the visual cortex. The structure of the physical cortex is quite understandable: it detects detailed features and integrate them in bigger concepts until you finally 'see'. But I think for more abstract things in the human brain we'd need more fine-grained data and the possibility to replay that data (in the future), so mapping and recreating the structure of a brain (digitally) will also be needed for when I am talking about "the ability to precisely debug the human brain."

What is funny is that AI currently serves as a very crude check to see whether we really understand brains at all. Just rebuild the brain in AI and see if it produces the same result. So part of this ability to precisely debug the human brain comes from AI itself. Since AI can be used as a hypothesis to test our understanding of the brain.

Couple of things: animal brains are cool too, AI can also progress without understanding the brain and this obviously isn't the only thing that will leap AI forward.

But if human brains become more debuggable (either through questionable ethics or technological advances), then it will benefit AI immensely.

Also the ability to have hardware that would be 10,000 times as fast and software that would be optimized for a 10,000 speedup would help. I know that sounds a bit clunky but it does.

I think what the theory embodied cognition tries to point out is that strong AI is not this cognitive "thing" that can be transferred from container to container. Imagine that we discovered that Dolphins were just as smart as us and also imagine aliens discovered us, embodied cognition says that the cognitive apparatuses in each of these and us is radically different because brains are always wired up to bodies.

It's not that AI needs to be embodied (computation and cognition are always housed in something), it's that what the housing is will affect the AI. In other words, don't think that strong AI means "thinking like a human" because that AI won't have a human body.

At least that my take on it.

I am not so sure that it is strictly necessary. But i also like the theory.

Here is an interesting, if dated paper i just read [0]. Not so much that it needs embodiment, but that it needs to be trained on the real world.

[0] https://people.csail.mit.edu/brooks/papers/representation.pd...

I've been studying Embodied Cognition AI and going to conventions since around 2009, it started seriously gaining popularity around 2012-2015, but then you get division between the new followers and the old about how extreme and radical you need to be.

The new people accuse the old of being too hand-wavy and airy fairy, the old people accuse the new of not taking the new ideas seriously enough, and not accepting the criticism of their entrenched views. From this comes progress.

For my money, the best philosophy comes from Dan Hutto, best book being Radicalizing Enactivism (Hutto and Myin, 2012). The best neuroanatomy with regards to consciousness and intelligence came from Walter J. Freeman III, best book being How Brains Make Up Their Minds (Freeman, 1999) and the best up-to the minute AI research is from Tom Froese. See "Referential communication as a collective property of a brain-body-environment-body-brain system: A minimal cognitive model" (Campos and Froese, 2017), and his (personally very interesting) work on the possibility of self-organising governance in Teotihuacan.

If you just want to have an introduction to the distinction between the two approaches to AI then you can do no better than read the snappily named paper "Why Heideggerian AI Failed and How Fixing it Would Require Making it More Heideggerian" (Dreyfus, 2007). It's true this paper appears to skip straight from Symbolic GOFAI to radically embodied dynamical systems, skipping Connectionism, but the issues raised in the paper can easily be used see that neural networks will fail to reach anything like intelligent behaviour unless they begin to draw strongly on the embodiment literature.

I can see from the dates of my recommended publications that I've not been keeping up particularly well, but I've been writing up my thesis on a slightly different subject.

Pretty sure embodied cognition was very much a thing in the mid 90's.

It seems we keep moving the date forward with AI techniques, claiming they are newer than they actually are, are we in denial?

I think its not too controversial a statement to say that Embodied Cognition was kicked with by the publication of The Embodied Mind (Varela, Rosch, Thompson, 1991), but I noticed a significant increase in its popularity in the 2010s.

fair enough :)

In my view, intelligence is an information processing problem. Neural Networks are situated in an environment to some degree. You have inputs, outputs, and then some measure of error with respect to the outputs and some objective function. The objective function can be seen as a way of encoding information about the world in which the NN is producing outputs, and provides the basis for the feedback loop that is used to train the NN ( forward-propagation and backward-propagation ) and produce a representation of the world.

I think these theories are great, but unless a theory makes a mathematical argument about information processing, I think they can be highly misleading and confusing. Natural language has been tripping up philosophers for a long time, and I think the lesson has been learned that we must make mathematical arguments if we ever truly wish to get to the bottom of something.

https://en.wikipedia.org/wiki/Integrated_information_theory

Thanks!

Well, once anyway. Then you copy that to every elevator and toaster controller, and voila! It understands your need for unburned toast early in the morning.

As if humans haven't also hit the barrier of meaning? And yet we've still made unimaginable discoveries. I almost added the word "progress" but I guess that depends on what you find meaningful.

The article states: "But ultimately, the goal of developing trustworthy A.I. will require a deeper investigation into our own remarkable abilities and new insights into the cognitive mechanisms we ourselves use to reliably and robustly understand the world."

Why limit the field to the capacity of humans? What the author calls "remarkable abilities" and "robustly understand[ing] the world" can also be seen as just reproducing our own innate and learned collective human biases. Our theories from observation, and their unprecedented ability to predict future events, is more about describing the world vs understanding it. Is there any topic in the world that doesn't have contrary interpretations?

What quantifiable metric would we even use to gauge artificial intelligence's grasp of "meaning"? We don't even have one for our own.

This is not philosophy. This is "apples are smaller than watermelons", "objects must be contiguous to be the same object", "the thing that makes an elephant an elephant is the presence of several large key features (ears, nose, trunk, legs) and not its color". If machines are going to meaningfully process our data, they're going to have to agree with us on certain base facts of reality.

> What the author calls "remarkable abilities" and "robustly understand[ing] the world" can also be seen as just reproducing our own innate and learned collective human biases.

From the article:

> “The bareheaded man needed a hat” is transcribed by my phone’s speech-recognition program as “The bear headed man needed a hat.”

If A.I. is going to work with humans, then yes, reproducing some of our "biases", such as the tendency to describe someone as "bearheaded" far more often that "bear headed", is a good goal. In an alternative universe of chimeras, a different bias would be needed.

But in any case, the A.I. needs to "understand" that one of these transcriptions is much more likely to be correct than the other. So a level of processing beyond "sounds like this word" is needed.

Take the famous absurd quote++: "Time flies like an arrow. Fruit flies like a banana." Reading the second sentence makes you, a human, double-take, and re-read the first sentence, and try reinterpreting it to see if it makes sense a different way. (The fact that it doesn't is what makes it funny.) An A.I. that has no "wait did that make sense?" step, no "is that funny?" step, is going to produce results that disappoint or confuse us humans who do.

(++Interestingly, this quote seems to originate in A.I. research: https://quoteinvestigator.com/2010/05/04/time-flies-arrow/)

Good article. I think issues like the mistranslation "The bear headed man needed a hat" really show that what we need is not an "AI algorithm" but AI agents. The lack of common sense that we see in machines is very much intended, it's what makes them machines in the first place.

Humans don't make these mistakes because humans are able to create stories and place even a mundane translation in the deep context of a 'world' that must be coherent. Heads simply don't have bears on them, we wouldn't trust that translation even if we had heard it clearly, and we'd rather think we hallucinate before we'd drive on a road that goes nowhere in an impossible direction.

Algorithms that are just glorified feature extraction machines, in my opinion, can never create a coherent story, so I'm very skeptical about the claim that we will have somehow solved intelligence in the next ten years. It seems to me like we are almost nowhere closer to it than we were decades ago.

Isn't it also true that what we define as coherent is based on observed data that could be fed into these algorithms?

In a sense you can always assume that the answer to these problems is just 'make the neural net bigger' but I find this deeply unhelpful for two reasons:

1. This clearly does not seem to be the way humans learn. Humans can learn from very few examples, in entirely unguided environments, and they don't face the same issues that existing algorithms suffer from. (for example humans have no big problem with rotational invariance, whereas ML vision algorithms do).

2. It's essentially surrendering to the fact that we aren't able to understand how cognition works and build higher-level representations as a result. The goal of AI research can not just be to feed data blindly into enormous primitive structures, it must also be to get a grasp on what sort of complex structures are part of intelligent agents and how they interact.

Humans can learn from very few examples, in entirely unguided environments, and they don't face the same issues that existing algorithms suffer from. (for example humans have no big problem with rotational invariance, whereas ML vision algorithms do).

That's because, contrary to the zeitgeist, humans are not a blank slate. Our brains are the result of billions of years of evolution. They are extremely well adapted to modelling the natural environment and the behaviour of other beings around us. This is in stark contrast to computers which we start from nothing and force feed a huge amount of data without context and then expect results. The fact that this approach works at all for some tasks is staggering.

Not an expert, but as long as there is no way of knowing WHAT it is the algorithm is learning it seems to me that it could never work reliably. It might look perfectly reasonable until you hit one of the triggers the algorithm used to segment the data.

Someone somewhere shared a story about using machine learning to spot the difference between US and Russian tanks ; which apparently worked fine until field testing, where it failed miserably. What the algorithm had learned was the difference between great quality photos of US tanks and poor quality photos of Russian. True or not, this is exactly the kind of issues that will keep popping up.

Plenty of people are spending plenty of time figuring out how to mess with facial recognition as we speak by taking advantage of the same fundamental weakness.

Oh, yep! It's super simple to induce systemic errors like that. Take https://github.com/kevin28520/My-TensorFlow-tutorials/tree/m... . Lighten every dog by 20%. Darken every cat by 20%. Train. Take image of cat, lighten 20%, watch as it's transformed into a dog!

For large corpora, it's impossible to know what features got selected. They probably aren't any feature a human would consider.

I agree with the general skepticism regarding AGI, but I think it's worth pointing out that there is some progress towards robust representations of abstract conceptual knowledge (http://science.sciencemag.org/content/331/6022/1279) that are far beyond feature extraction.

> Heads simply don't have bears on them

https://encrypted-tbn0.gstatic.com/images?q=tbn%3AANd9GcSzPe...

Perhaps the man proceeding towards the grizzly had a negative economic outlook.

There are literally hundreds of word-sense-disambiguation papers being published per year, and have there have been for about a decade. The state-of-the-art has been advancing relentlessly, and examples like the ones cited here are being zoomed past right now by the latest thing, which is even larger models pre-trained on unsupervised data. There will be a next thing, and for NLP this will be backgammon or chess to 2020s Go.

Embodiment is a better criticism, but this article is still bad, and is written as though Intelligence were some essential soul-like property that agents either possess or don't.

It is really amazing how often articles with this viewpoint repeat one particular trope: they put the word "understanding" in italics. Look out for it.

But if you do an error analysis on even ShotgunWSD (Betnaru 2017), you get errors that humans would not produce - archaic favors, missing recent terms, confusing opposing meanings, ... When the average researcher can easily select the correct sense. And that's a DETERMINISTIC algorithm! Swapping ELMO for pure WordNet gives us slightly better results in that sense but introduces literal random chance errors. And the older, Bayesian models are worse!

While we'll probably solve WSD to a high degree of accuracy (such that engineering bugs cause more failures than the model), our current approach is still fundamentally flawed. We need a better method of imbuing causal relationships than correlation+chance.

> archaic favors, missing recent terms, confusing opposing meanings,

Humans make errors like these all the time (e.g., I don't know what you mean by 'favor' here, and I'm sure many people wouldn't have the type "ELMO"), and the dictionary definition of the word shifts accordingly, except we can't really call them 'errors' when it's a natural phenomenon that we're trying to match. Humans make seemingly random errors sometimes too, confusing one word for a mother, I mean another. When we do it we infer (perhaps correctly) deep psychological causes. Current machines don't have personal identities and desires and goals like we do, but embodied agents with a natural selection process could have those things, there isn't an obvious technological barrier. And if you build in a language model that shares some of the general computation machinery we could reasonably expect it to make those kinds of errors.

> Heads simply don't have bears on them

Also, humans may refer to other humans as toy-bears and get punished for it (https://www.theguardian.com/world/2018/aug/07/china-bans-win...). Human language is hard, metaphors are hard.

The wild Zuckerberg quote aside, I don't believe many people expect this current wave of AI to approach general intelligence. I would be surprised if we weren't all mostly in agreement with the author.

In my experience, domain practitioners who spend our days elbow-deep in tensors are well aware that what we call AI right now has nothing to do with AGI.

That knowledge does not seem to generalize well, though. Even technically literate people who aren't in the trenches seem to have frankly insane ideas about AGI. This phenomenon is not helped by armchair philosophy, the frothy chatter of the tech-adjunct world, or the naive idea that "well, transistors are easy, so brains should be too!" that is all too common in people who haven't stuffed electrodes into brains and then spent weeks figuring out what the hell could be going on in there.

You'd have a pretty hard time figuring out what a CPU does if all you could do is poking a small number of needles into it, cutting it into slices to study under a microscope and measuring which parts get hot while you use it.

https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...

true fact.

The thing is that progress is so fast in some areas of ML, you dont really know when something could emerge. Who knows, maybe with enough training data and a deep enough network with the right supervision, something extraordinary pops out.

Google google bert nlp. Google recently released an NLP algorithm that basically beats all predecessors, but the algorithm itself is extremely simple. Just lots of data and compute, no telling what will happen in ten years when the amount of data has exploded and GPUs are cheap.

This mentality is exactly what hprotagonist is talking about. All signs currently point to AGI being far away - at least several breakthroughs in currently unknown unknowns away, in fact.

If the most you can say about the likelihood of something happening is that, "there's no telling what can happen in the next ten years!", then the only informed forecast is that it won't happen. We can likewise say we don't know if an extinction level event will happen due to a meteor (that we couldn't detect for some reason) hitting us in the next decade.

Every time someone makes a sobering statement grounded in reality about artificial intelligence, another comment pops up which reduces to, "but who knows!" It doesn't mean anything.

bert provides incremental improvements on tasks that don't really stress-test contextualized world knowledge in linguistic tasks — contrast with Winograd schemas (SOTA: https://arxiv.org/pdf/1806.02847.pdf). Corpus data and compute don't magically solve everything.

The paper you linked to shows how a simple DL brute force approach (large model trained on lots of data) "successfully discovers important features of the context that decide the correct answer, indicating a good grasp of commonsense knowledge."

Which kinda goes contrary to your last sentence. Go figure.

My point is that while the DL + large corpus approach is the best available, it's still way, way lower than human performance (Tables 4 and 5).

Sure, but we are talking about how fast it's progressing. Given that NNs for NLP field pretty much started with Bengio's paper in 2003, I'd say it's accelerating at an amazing rate.

But NNs for language processing have been around much longer than that, e.g., Elman, 1990: https://crl.ucsd.edu/~elman/Papers/fsit.pdf.

Yeah, Bert's great, but it has the Google Paper problem in that nobody else can reproduce the results because nobody else has the compute to train the dang thing. It also doesn't solve the fundamental issues - it scores a few percent higher on some standardized sets, but it gets wrong answers that are just as explainable as any other system - shrug-emoji. ‍️‍️

A couple months ago, I was on an NPR show trying very hard to refocus the discussion on real near-term threats caused by AI: possible job loss as well as enabling authoritarianism, predictive policing, and quashing of dissent. They wanted to talk about AI coming to get us all. (Evil) AGI seems to be a more exciting topic.

"...enabling authoritarianism, predictive policing, and quashing of dissent" sounds adequately evil to me, but - you know - I get what you're saying.

Can you recommend any good reading about these threats?

Do you have a link to that show? Seems like a good one to hear.

I think this is the show he was referring to: http://www.wxxinews.org/post/connections-did-2001-space-odys...

Okay, fine. I wish more people talked about Jeff Hawkins work at Numenta or cortical.io or sparse distributed representations. But in particular, Jeff's vision of understanding the neocortex is really cool stuff. If I could cast off the shackles of having to work for my money like a filthy commoner I think I would lean towards learning a more biology inspired approach like Numenta is doing.

I see plenty of ML researchers in person and online who think we are getting closer to real AI and it just requires some evolutionary improvements over current techniques.

Sure, but current techniques are a long ways from that.

Well, quite a few C suite folks seem not only to believe it, but to be convinced that anyone who disagrees is effectively committing sabotage. This is because every vendor is telling them this is so, and offering to sell it to them. This is dreadfully damaging, and a time of reckoning will come, but don't expect justice.

I think pop-sci fuels overzealous misinformation in even the tech-literate. A lot of innate, unconscious cognition is underestimated despite its complexity. E.g. consider the ability of humans and most other species to approach, navigate, and understand arbitrary space and its objects - you can't do them same with AI without extensive training and covering corner cases. And even then, a flying insect such as a dragonfly will outperform your model.

Everything OP says is correct.

And let's not forget that even in the absence of adversarial attacks, even with flawless associative mapping and incredible generalization, even hypothetically solving the problem of higher level ontological 'meaning', there's STILL absolutely no known way for AI to even begin to address the matter of the qualia - how to build something that's actually has a conscious perception (of pain, for example). We're groping in the dark without matches, and there will be many more seasons/winters to come.

Qualia is a red herring, just like the p-zombie topic. Consciousness is a program run on your brain to make you think you are conscious. (And as long as this explanation works and is simpler than all the others, we should go with it, instead of claiming how incredibly hard and unfathomable consciousness is.)

What's hard is bootstrapping the whole thing. Evolution simply used brute force and copy-pastes a sort of working, already pre-wired, fine-tuned mishmash of faculties as a brain from the previous generation to the next. And uses the old gen to train the next gen.

> Qualia is a red herring, just like the p-zombie topic.

It is NOT a red herring ~ it is a real problem for reductionist physicalism. However, because it cannot explain qualia, it tries to ignore, dismiss, and belittle, the enormity of the problems presented by qualia for its attempts at trying to explain.

> Consciousness is a program run on your brain to make you think you are conscious.

This is a presumption with scientific evidence. This is merely reductionist physicalist philosophical dogma.

We can be certain that we are individually conscious and aware ~ but not necessarily why or how.

Consciousness cannot be reduced down to a bunch of changes of the brain's matter. Consciousness is qualitatively different to how the brain functions, even though consciousness can indeed be influenced by the changes in brain states. Even though literally no-one knows how.

> (And as long as this explanation works and is simpler than all the others, we should go with it, instead of claiming how incredibly hard and unfathomable consciousness is.)

It is not simpler ~ it makes a set of presumptions which reductionist physicalism must explain scientifically. And even now, no scientific explanations are forthcoming. The philosophy has had centuries to explain how consciousness arises from the material and physical, and yet no explanation has been offered as to how the brain can magically create consciousness.

There are literally no models at all that demonstrate, solidly and irrevocably, how particular configurations of brain matter directly translate into consciousness.

> What's hard is bootstrapping the whole thing. Evolution simply used brute force and copy-pastes a sort of working, already pre-wired, fine-tuned mishmash of faculties as a brain from the previous generation to the next. And uses the old gen to train the next gen.

A bunch of presumptions and just-so hypotheses without scientific evidence.

> It is NOT a red herring ~ it is a real problem for reductionist physicalism.

No, it's not.

> However, because it cannot explain qualia, it tries to ignore, dismiss, and belittle, the enormity of the problems presented by qualia for its attempts at trying to explain.

Qualia has no objective existence, and as such isn't even in the domain of things science concerns itself with; an observer can (if possessed of qualia) observe their own, but cannot observe, test, or be materially affected by its existence in others.

> There are literally no models at all that demonstrate, solidly and irrevocably, how particular configurations of brain matter directly translate into consciousness

There's literally no objectively verifiable evidence of consciousness or any features it might have, so there is literally nothing to model, and, even if one somehow came up with a model, no way to validate it. Qualia, insofar as it can be said to exist, is simply irrelevant to any empirical model of the universe.

Does qualia matter? Metaphysically, perhaps, but materially, no.

> Consciousness cannot be reduced down to a bunch of changes of the brain's matter.

This is a claim, show the proof then.

> Consciousness is qualitatively different to how the brain functions,

Again, you need to prove this. First of course define what is what and why they are irreconcilable.

We have a consistent theory of the world for the small and large, from nuclear physics to cosmology, and everything in between, and none of them require extraphysical things. Why consciousness would be different?

> and yet no explanation has been offered as to how the brain can magically create consciousness.

I just offered one, and there are probably countless other explanations. The problem is usually testability not abduction of explanations.

> There are literally no models at all that demonstrate, solidly and irrevocably, how particular configurations of brain matter directly translate into consciousness.

We don't have to, we can test and weight hypotheses and eliminate them in other ways.

So far you have offered nothing that would help eliminate this hypothesis (counterexamples or other arguments would be great).

> there's STILL absolutely no known way for AI to even begin to address the matter of the qualia

Who, other than metaphysically, cares? I can't observe, test, know or be in any way materially affected by whether or not you experience qualia, it certainly makes no material difference whatsoever if an AI actually experiences qualia.

Discussing qualia has about as much relevance to anything as discussing how many angels can dance on the head of a pin.

You might not see my other comment, but you couldn’t be more wrong. We may not understand qualia, but it is indisputable that qualia provides us with additional data that affects our bahavior. It provides meta-data about those who also experience qualia, which enables at the very least greater predictive powers regarding those who experience qualia, and at best, enables empathetic behavior.

It speaks to empathy, as already mentioned.

Pain and pleasure, which are qualitative experiences, are the basis of operant conditioning (unsupervised learning ala behavioral psychology). This covers the massive domain of behavior that's informed/dictated by reinforcement/punishment. So they are pretty fundamental if you care about building something that's analogous to biological life, depending of course on how deeply you want that analogy to extend.

> Pain and pleasure, which are qualitative experiences, are the basis of operant conditioning

The behaviors of attractive and aversive reinforcement are the basis of operant conditioning. The whole issue about qualia is that exhibiting behavior to which subjective internal experience is usually attributed does not demonstrate the subjective internal experience. The people arguing for the importance of qualia and it's unattainability for AI aren't arguing that it is particularly challenging for AI to exhibit attractive and aversive reinforcement, or any other external behavior, they are building a castle they can retreat to in the face of any observable behavior.

You haven't actually said anything I disagree with, even though I'm in the castle. "Retreat" sounds a little harsh. I would say "Comfortably ensconced". Even if an AI nails down the behavioral/functional aspects of a biological brain, that doesn't necessarily mean that the qualia have come along for the ride.

Does it matter if they have? That seems to be the real question.

Allow me to propose a thought experiment: I'm a traveller from the future, where the notion of 'qualia' is understood and engineered (maybe using techniques we would somewhat recognize today, maybe not). I present you with a machine that has a single red button. When the button is pressed, there is no observable behavior, except that 10,000 AI's are instantly created and subjected to extreme agony until the button is pressed again. What are your thoughts on pressing this button?

> Allow me to propose a thought experiment: I'm a traveller from the future, where the notion of 'qualia' is understood and engineered

This is incoherent: qualia by definition cannot be engineered, or even validated by external observers. This isn't a technical limitation that can be overcome with progress, it's inherent in the definition.

Maybe the future involves direct brain interfacing between humans via biological interconnects, and a new kind of engineering will evolve, defined not by behavior/functionality in the objective world, but in a realm of shared consciousness that is only subjectively accessible to biological entities that have a similar level of brain evolution.

In fact our own brains probably already work this way, with different parts of the brain acting as loci of lower-level consciousness, but also somehow becoming more than the sum of their semi/sub-conscious parts when acting in concert.

And maybe computers just can't participate in this shared consciousness, even in principle, because they're made out of the wrong 'stuff' along a dimension of reality we don't yet understand (or may never understand).

The only solution to human-friendly AI is to instantiate a quaila-producing consciousness.

Qualia is, by definition, data that non-conscious entities lack. We don’t have to understand what qualia is or how consciousness works in order to train software to develop both. All we need to do is understand how the qualia data affects behavior. If we train toward those unique behaviors that only qualia can provide, then ML will figure out how to produce consciousness and qualia on its own.

What is that behavior?

Empathy.

Empathy is knowing what it feels like to subjectively experience qualia. I know the feeling of awe when I see something beautiful. I know the feeling of pain when I lose a loved one. I know the feeling of jubilance when I succeed. And because I know what it feels like, I know what it feels like for another. And that is real data that allows me to understand what another person will say and do, and to understand what I should say and do with respect to their state.

This is real data, and luckily it doesn’t take a super-intelligent AI to produce. See your dog (and perhaps cat). A dog only understands your feelings because it too has those feelings.

We don’t need to train for intelligence to solve the question of meaning. We need to train models that demonstrate empathy.

I’m not aware of any research in this area. But Douglass Adams’ happy sliding doors and depressed androids were on point.

Douglass Adams! So ahead of his time. I remember the elevator with precognition - so it would know ahead of time which floor to be at, and already be there when you hit the button.

But putting a super-intelligent being in a box that had a lifetime limit of travel over about 1 linear block in a dark tube, made it a bit strange after a while.

It all comes down to the fact that we don’t really understand what intelligence is. We know it is an emergent property of several mechanisms, working together, and we are even getting a decent grip on what those mechanisms are and how they work, but we are still struggling to see the big picture of how intelligence actually emerges.

We don't know if it is an emergent property. That's what we think it is likely. but we won't know until we find out what consciousness is. And we won't find that out until we crack the secrets of the brain.

There once was a time when we thought our bodies moved because there was an "emergent property" called the soul that instructed our bodies to move. It took a very long time for humans to discover how our bodies through the science of physiology.

"The soul" is (was?) explicitly not an emergent property.

Aside from the technical obstacles currently barring us from realizing AIs capable of human competency, we should be seriously considering the social question of whether or not such a technology is something we really need or want.

The world is already saturated with technical systems and artifacts that have effectively flattened social relations, eradicated traditions, and generally reduced the field of humanitarian meaning and theory to nil. All our problems have become technical problems. We’ve even reached a point where we try to solve social problems with technical solutions, and when that doesn’t work we try to fix the boo-boo the naive and rampant application of technology has caused with, guess what, more technology.

The philosophy of technology is perhaps the most important discipline of our time—there’s little left in life that technics hasn’t in some way enveloped, either insidiously or openly.

Those seem mostly like arbitrary value judgments based on what you personally happen to be comfortable with. Ask more fundamental questions. Are we necessarily better off with traditions than without?

Sure. Imagine a world in which all tradition and culture was wiped out. Imagine a world in which all technology was wiped out. They're both absolutely horrible. And certainly, some traditions are better discarded, just as some technologies (the A-bomb being the prime example) are better left unexplored. I'm only trying to point out the tendency of our era is to forget humanitarian values entirely in the face of 'objective', 'practical', 'utilitarian' and technical progress.

I am making a value judgement, and your dismissal of my points simply because they entail value judgements is exactly the sort of attitude that has led to the dissolution of culture and the near reemergence of fascist and populist thought in arguably the most technologically advanced country in the world. There's a reason technology couldn't stem the reemergence of such thinking, and in fact the social conditions rampant, unquestioned (or if so, not nearly enough) technological development has caused has contributed to the revival of such thought. Such an approach annihilates dialogue before it can begin. Living social subjects no longer endeavor to understand each other because they fall back on a ideology that tells them they it's "above" values.

I am championing the philosophy of technology, and I am somewhat familiar with it, but I also genuinely believe it's important, and yes, that is a value, and one that I stand by. The sort of world I'm hoping will emerge in the future is dictated by those values. I just hope there's still enough folks in tech considering the future their current values might create.

I recently realized that meaning cannot be done using pattern matching only, which is what (current) machine learning is about.

The key insight came from article on bullshit: https://news.ycombinator.com/item?id=17764348

I believe understanding (things having meaning) comes down to whether you can produce an example from the given description of the situation. If you can produce an example, then you can say you understand the description.

The problem is, you can have situation descriptions that are contradictory (have no examples, thus have no meaning) yet they are arbitrarily close to situation descriptions that have examples (and so have meaning).

A good example are those Escher-like impossible objects, which look very much like real objects, but humans can easily see they are meaningless (they cannot be interpreted, and thus imagined, in 3D). Another good examples are sentences from the above paper. The bullshit sentences are the ones for which you cannot construct a mental example in your head.

I suspect this happens for the famous flaws in deep learning as well, the deep learning network cannot learn that some inputs are contradictory.

I believe that this is actually ultimately related to the boolean satisfiability problem. In theory, one could determine whether some learning agent is only using pattern matching or is actually having understanding (is able to recognize logical inconsistency in the input) by teaching it different SAT instances, and the agent that would be able to learn the difference between arbitrarily similar SAT instances with and without solutions could be considered to have understanding.

I've started thinking about the whole "do computers grok meaning" thing in terms of proof. If a computer tells me a true theorem, I don't know if I can ascribe "understanding" to it. If it tells me a true theorem and comes up with a proof, then I'm comfortable defining that as understanding within the context of that particular proof language. It passes the buck up a step. You define a new notion of understanding-in-a-given-language and then get to argue about whether that language/deduction system constitutes true "understanding."

The problem isn't deduction. It's getting things into the deductive system in the first place. You have to model the world, and your model will probably be incorrect somehow (https://www.student.cs.uwaterloo.ca/~cs492/11public_html/p18...).

Okay, but what if it just made stuff up by trying everything until something came out true? We have theorum provers on this model today. Does such a system "understand" mathematics? It's probably more explainable than anything else, but the strength of the current generation of learning is application to cases with ambiguity present, and our current approaches here are completely unexplainable.

I'm saying that we can define what constitutes understanding by defining what constitutes reasoning within a given inference system. That's a simpler task, but doesn't solve the original problem. It just passes the buck. Now instead of asking whether a given learner performs reasoning and understands, we could ask whether the language/logic constitutes understanding.

I'd definitely say that the current theorem provers have some form of true understanding, even though it isn't the same form as we have. For the typical ML learners, I think it's more interesting to ask and taxonomize "in what ways is this reasoning/understanding" rather than just ask "is this reasoning/understanding?"

'Understanding' is a matter of degree. Humans do not know very well how their own brains generate ideas, thought or language. Yet they appear to be capable of understanding.

Turns out the primary use case of AI at the moment is duping low info voters into electing fascists. AI has an IQ of 87 in narrow situations, while the targets are working with 85 and below.

One foot in front of the other, but the chasers got a little bit ahead of the horse here.

Didn't check but, considering there are probably thousands of "small tasks" that are being solved by AI, can one build an AI that can choose the right "small task" to apply to a never-seen-before problem ?

I agree with the author overall, but I think she picks a poor example for dictation.

“The bareheaded man needed a hat” falls squarely in the domain of contextually-aware models and would likely be transcribed correctly using deep learning.

Optimistic framing: "Machine learning algorithms, and domesticated animals such as dogs, don’t yet understand things the way humans do — with sometimes disastrous consequences.

This is the reason why chatbots did not live up to the hype. It is hard to have an agent that offers solution in a domain as well as being generally conversant.

Here's what I've realized both about the universe AND intelligence .

Silicon cannot touch either of them. I didn't always think this though. I thought intelligent beings, but first smaller structures, could evolve in a large digital universe. I made a blog post: http://scrollto.com/blog/2017/04/11/life-a-universe-simulati...

It took a long time but I eventually created what I set out to do.

Here is ScatterLife: https://github.com/churchofthought/ScatterLife/

But the result was far from what I wanted. Even with a Titan V, a world of only 4096×4096 could be handled at a reasonable update rate. I basically had spacetime foam under the ideas of Doubly special relativity, ie. Feynman checkerboard universe.

https://en.m.wikipedia.org/wiki/Doubly_special_relativity

https://en.m.wikipedia.org/wiki/Feynman_checkerboard

If our own universe is digital, then it updates at C/plank_length times per second, over 10^40 hz. Not only that but it consists of over 10^185 planck lengths.

Nothing interesting is really viewable or even extrapolatable from the smallest of truths or fundamentals. Thats why many predictions of string theory need higher energies to be tested. The smaller you delve, the higher energy needed.

In any case, I realized that evolution itself has been fueled by orders of magnitude. Symmetries of matter and energy, planets stars and solar systems...same magnitudes required.

Even the best silicon isnt going to have 10^20 transistors on it. We would have to somehow go analog..use chemicals or matter in a way that didnt require slow refining and construction. Chemical based computing....

Now about ML and AI, same issue is present. Brains have 100 billion neurons. The best GPUs have maybe 5000 cores.

The only way forward is to maximize what each core does - as much as possible.

I learned this with the cellular automatas. Black and White 2-state automatas are neat but are a huge waste of processing power. They hold little information. Better is integers. Even better is floats. Why stop there though? Lets use complex numbers.

https://github.com/churchofthought/HexagonalComplexAutomata

Magnitude is against carefully crafted silicon. If we want to achieve what magnitude can, with silicon, we need to make sure our fundamental neuron units arent unnecessarily sparse. Magnitudes can afford to use simple units. Silicon cant. 5000 maybe 100000 cores when advances in fab accelerate. But still not 100 billion. Still needs to use the advanced abilities of the cores to their full extent. A neuron cannot compute any universal function...but a gpu core can.

Anyways, I am almost failing to mention that I dont believe we have anything to worry about. Unless more research is devoted to special purpose hardware (consider an i7 has 731 million transistors), the software side will have a really hard time compensating for low magnitude.

Lets see what happens. Its going to be exciting none the less. I am doing ML work myself on Boltzman networks and RNNs snd Hopfield nets. This is a promising field and its emerging at breakneck pace. Cheers!

This is interesting. I too am interested in simulations and fields like Artificial Life where we start with some basic building blocks and hope to allow something capable of evolution to form.

My question is always "where do we start?" because, as you found out, starting with the most fundamental physics simulation we can conceive of, we are unlikely to generate much of interest for some time, if ever.

But I do have a feeling that in order to get something truly novel and interesting it's going to have to "evolve" in an "environment" and the challenge will be in identifying whether a particular instance of primordial soup is on its way to developing more complex structures.

I strongly believe that the importance of the multi-billion year process of evolution is seriously underestimated by the AI community and that it's pure hubris to think we can short-circuit that entirely and simply reverse engineer the brain with fancy algorithms.

> But I do have a feeling that in order to get something truly novel and interesting it's going to have to "evolve" in an "environment" and the challenge will be in identifying whether a particular instance of primordial soup is on its way to developing more complex structures.

I thought that way for a long time too which is why I chased the cellular automata ideas and eventually implemented many varieties. But nowadays, I think its all mostly torched by magnitudes.

The closest chance we have is not relying on the magnitudes. Instead of trying to evolve a universe, or variety of entities - we need to focus on a single entity.

I thought AI had a flawed premise, like you mention: attempting to develop single individual is directly antithetical to how life normally develops. None the less, my mind is changed since my automata dabblings. Single individual engineering is the only method that has the slimmest chance in hell.

Bingo.

Surprise! Meaning requires strong AI (and cultural understanding, much of which is only learnable from interaction).

Meaning doesn't 'require' strong AI -- the kind of meaning that strong AIs like us prefer to interact with does. There's plenty of simple meaning in simple systems, and they are quite useful to us that way. They just don't encapsulate the complexities of human experience very well.

The author is trying to make a point that AI is still truggling on basic understanding based on examples.

Google Translate renders “I put the pig in the pen” into French as “Je mets le cochon dans le stylo” (mistranslating “pen” in the sense of a writing instrument).

Well, Google is far from being the best at translation right now. If I try the author's example in Deepl for example "I put the pig in the pen" it translates into French as "J'ai mis le cochon dans l'enclos." which is the perfect translation based on context.

So, I still have to read the rest of the article, but right from the bat, you're wrong mate (or at least you provided an example fo something "impossible for AI" which is already perfectly done by properly trained/programmed AI).

Reply on news.ycombinator.com