Self-supervised learning: The dark matter of intelligence | Hacker News Reader

Self-supervised learning: The dark matter of intelligence

ai.facebook.com

179 pointsAnon8474 comments

One of the problems we have to be conscious about with self supervised learning is that the already existing problem of transparency and understandability gets worsened by orders of magnitude. If current billion parameter models are opaque, the result of SSL is, or will be, the equivalent of a black hole. How do you understand what the model has learned? Why has it learned that way? What things could have possible gone completely, horribly wrong in its quest to optimize its loss function?

This somewhat humorous but also worrying paper comes to mind: https://youtu.be/Lu56xVlZ40M

I think we really need to consider how we can mitigate the risk of raising completely ununderstandable yet scarily capable machines.

I dont agree with this, either we know what the pretext task was, or we fine tune with labels, but in either case, it's no different to analyze than supervised models.

I do agree that more parameters and a bigger training set makes the models more opaque (or at least more expensive to understand), but the self supervision is not the reason.

Or instead of analyzing the model we could analyze the training data for biases. Most bias comes from data.

Its bias all the way down...

Code, Data, Model, Use-Case, ...

I have this same problem with people. Just because we don't understand perfectly doesn't mean the output is not useful.

Sure, but humans are a relatively known entity. They exist with a level of variance that is not too extreme, and you can more or less, in a general way, predict their behavior to a tolerable risk level. We've seen billions of humans, we know what to expect of them. We might not understand how other humans work internally, but we understand their actionspace.

For AI models, that isn't really the case. We don't know how each individual model will act in the real world, and certainly not after having spent millions of hours training against itself or in a simulated environment. And as these models get more and more powerful, this growing uncertainty is something that I believe is worrisome.

> Sure, but humans are a relatively known entity.

We don’t know how brain works, why sleep exist, most of humans cultures and languages are not documented, hundreds if not thousands of psychological and medical things are unknown, etc. Machines and their applications are a few more magnitudes more understood than any human matter, by the sheer fact we created them. Their complexity is ridiculously low in comparison to human, their biology, psychology, etc.

on one hand i agree with what you're saying. we humans have done terrible stuff like wars, genocides, famines, destruction of ecosystems, extinction of entire species, etc. and that's only the things we did more or less deliberately. we might cause our own extermination or a mass extinction event "by mistake", and we don't even understand how basic parts of our minds work.

but all of that being said, i think it's also worth considering what a non-human entity with more power than us might be able to do. after all, even the worst examples of humanity are confined to at most "only" causing millions of deaths, but not the eradication of all life on Earth for example. it's not difficult to imagine that a non-human entity, with non-human goals, and with super-human powers, might act in a way that is contrary to our interests, or even to our existence.

This already applies to corporations.

AI is just the fruiting body of something that has already happened.

Corporate intelligence are still people, abiding laws and regulations for people.

Yes, I wouldn't want an army of self supervised learning military mini drones

That was not the original point. Humans are relatively predictable whether we understand how the brain works exactly. An AI model isn’t an actual intelligence in the normal use of the word. It’s a functional fragment that is able to emulate the ability to absorb a pattern of information for repetitive conditional recall in future processes. Until a researcher has had ALOT of time spent on observing the limits db particulars of a model, there are unforeseen behaviors it can have that can have incredibly real and serious consequences depending on the application. A more fair comparison would be training a new intelligent species to do something before learning about the characteristic behavior quirks of said species

Luckily we never have unforeseen behaviors with people that we then have to react strongly to after the fact ... We like to pretend that's true, but really, it isn't. Governments are built around the principle that single individuals and even small to medium groups can't be trusted to react reasonably, something that played out in human history again and again.

How many people have been in US prisons again? Oh right, about 3%. Clearly a high percentage of these people did something both unexpected that seriously damaged others.

So this is not a real difference between AIs and humans. One might even say, not a single AI has been convicted yet, so it might not be true for them. For humans, serious, bad, violent, illegal, even when totally irrational, ... behaviors are very common indeed.

(and frankly, if there ever is a true Human <> AI conflict, not allowing any kind of errors for AIs seems to me a strong contender for the casus belli)

Humans being unpredictable at the individual level although true does not invalidate the original point that employing a poorly understood neural net training methodology in critical systems of any kind is incredibly rash and presumptive

Then train the models on real world data? Verify outputs enough until confidence is achieved.

The computer can do whatever it wants. People can do whatever they want. The question will be what level of security access will they have.

The key difference today is people are really good at making rationalizations for individual decisions. Computers are not.

Sometimes decisions are generally important, when they are important they require trust, and if these models never generate a method of demonstrating success/trust they won't be adopted.

>> Then train the models on real world data? Verify outputs enough until confidence is achieved.

And therein lies the rub. Do people actually want to know the truth? They invent methods to obtain it, but is that their aim or is it to confirm their current understanding of the world? Unlike a human being that can be forced into silence through coercion or manipulation, a computational model, once proven with certainty, is never going to go back in Pandora's box. This contradiction of pursuing truth but suppressing the inconvenience of its conclusions hasn't disappeared for some reason despite greater and greater knowledge of ourselves. Will it ever?

Both? I'm not 100% following. I think people want to know the truth, change is slow and admitting mistakes is often looked down on. Humanity made significant changes based on what was learned in the past 100 years. How this will be used, and who will reap it's benefits is unclear.

Perhaps I wasn't very clear. Would the general public accept the truth of a data model that proved that a previously controversial/unacceptable idea was factually correct to the point it could not be denied by a reasonable person?

Several attempts at training with real world models have produced results that have been attacked for being "algorithmically biased". This isn't because there was anything experimentally wrong with the dataset or choice of model on their own, but because of the result's contradiction of a certain worldview. While, I agree with you that people come closer to the truth in the long-term, in the short- and medium-term, I expect certain people to pad their own data and quash the different findings out of ideological motivation for some time.

Very few individual humans can act in a way that affects hundreds of millions of people, computers can do that in a nanosecond.

Can't humans with computers affect hundreds of millions of people too?

Only with the computer's help, that's my point.

In a medical context, the doctor won't use it if he doesn't know where it gets its score from

As she should. Trust but verify. The biggest question is what will this be applied to. Without it we can't really discuss approaches towards developing that trust.

I'd consider The ability to output a rationalization in human understandable text (ie: english in my case) will determine success /failure.

Not sure I see your point.

How is it harder to understand a model that was trained to predict which two images crops come from the same source image (a la contrastive loss for example), than a model that was trained to predict which class an image belongs to?

What's the algorithm?

> How do you understand what the model has learned?

How do you do that with humans? You watch their behavior, language and you test them.

Yes. Humans make mistakes, humans with computers make mistakes faster. Computers without humans make mistakes even faster.

Humans can explain their reasoning when you ask them. The best we have with NNs is a fancy version of first-order sensitivity analysis.

Humans cannot really explain their reasoning. It has been shown that they usually invent an explanation that matches what has already been decided in their black-box.They mostly use post-hoc explanation, but cannot explain the true decision mechanism.

If you make them believe that they choose another decision, they will create another explanation on the fly.

This! Decision making is mostly "irrational". I wish it would be different.

Decision making in humans is a mix of slow and fast thinking. Fast thinking - like reading, writing, physical reflexes - requires little effort and is best suited for repetitive situations.

Slow thinking - such as reasoning - is more expensive so we only use it when we are faced with a new situation for which we have not 'cached' the answer. It entails the creation of a mental model and imagining outcomes. The exact mix of the two modes is tuned by evolution.

If all we had was the slow system we'd be much less efficient at survival.

Tools for exploring the models are already being developed.

Language models that are not given concepts of part-of-speech, dependency trees, grammars, develop representations of their own and people have found ways to inspect them.

Could you please provide pointers to relevant papers? I'm very interested.

Here's a paper on how BERT (a large Transformer model trained using self-supervised learning) implicitly learns the traditional NLP pipeline: https://arxiv.org/abs/1905.05950

I wrote a book about this 10 years ago:

https://arxiv.org/abs/1104.5466

Annotation doesn't scale. You are not going to get to real intelligence by labelling billions of data points.

I expect that computer vision for autonomous vehicles will become good enough for practical use quite soon after the AV companies start doing large scale self-supervised learning (I don't like that term). Maybe Tesla and Waymo and others are doing that already.

Another great application area is medical imaging. Progress in this area has been seriously hampered by the limitations of the annotation paradigm. Once people start to do SSL, there should be rapid progress.

State of the art performance is being broken in multiple fields rapidly these days. However, AI explainability has a long way to go. Scaling opaqueness makes this problem worse.

Fine tuning labels black-box style is a terrifying concept to most who are working in fields where great risk must be managed to avoid unintended and disparate impact. Facebook making an oopsies suggesting my friends face in a photo instead of mine with a SSL trained model may seem trivial, but this non-human oversight is concerning for other applications.

If you were rejected by a bank for a home loan, you would want to know why your creditworthiness wasn’t evaluated positively (and banks must explain precisely why to regulators). And if a self-driving car made a decision in a collision, or a CV model performing cancer screening in X-rays that gave a bad result - ...or recommending politically divisive content... - or a thousand other real world examples that have human impact.

High risk models should work with humans in the loop, not autonomous.

Would this rule out self driving cars from the get go?

Depends on what you mean by self driving cars. To me it is not even clear that driving assist tech will ever be advanced enough for cars to drive themselves reliably.* But that aside. Human in control + driving assists is already safer than fully manual driving. The question is whether the opposite is even safer. It is obvious to me that self driving with the human as backup is a joke. Humans cannot react fast enough in real emergencies. So if you want the car to mostly drive itself, it has to actually always drive itself. And to me it is not clear whether this is really going to be safer than the human+driving_assist scenario.

*Of course I am talking about tech based on the current DL approaches. If we had AGI, then the question does not even need to be asked.

Yes, this is the point I am trying to make. Self-supervised models using already opaque techniques built by an ethically flexible company is not a good recipe.

A sufficiently complex algo black box could fool the people.

We want our models to be opaque in the same way we like to ask people why they made a certain decision or act in the way they do. What's interesting to me is if you want to know about a person you'll get better information by asking their closest friends than themselves.

Perhaps there's something in the black box nature of our own self understanding similar to these hyper complex function approximators.

Judges make harsher rulings when they're hungry. A regulator might decline a loan and justify it b/c they had a bad experience with someone that reminds them of the person they're dealing with. AI makes bad decisions in boundary conditions or when they're trained on biased data. I think it's good to strive for opacity in every case but in complex decision spaces I'm not sure if we'll ever get fully satisfactory explanations.

Which is a meandering way of making the non-point that models making judgments have made me re-evaluate what I trust and why, and I think in many cases I'd rather trust a black box if I know what's gone into it and what's come out.

> Judges make harsher rulings when they're hungry.

Actually, that finding is highly improbable: https://nautil.us/blog/impossibly-hungry-judges

Everywhere you wrote opaque you meant transparent.

I don't really agree with you.

> If you were rejected by a bank for a home loan, you would want to know why your creditworthiness wasn’t evaluated positively (and banks must explain precisely why to regulators).

This is why we don't use humans to evaluate credits, but precise algorithms. Humans are just applying the algorithms, they are not evaluating themselves with their gut feeling. I don't see why this would change with AI. Regulations prevents unexplainable tools to be used there, so deep learning black box models will not be used, similarly to why humans are not used today.

But in cases where performance is required, but not explainability, then deep learning will strive. And I believe that cases where explanation is required is only a very small subset of areas where AI could be useful.

> And if a self-driving car made a decision in a collision

Would you prefer a car that crashes once every 100 million miles but is not explainable, or a car that crashes once every 100 miles but is interpretable can explain why it crashed ?

Isn't it interesting that an industry like banking, built on those beautiful, precise algorithms, blows up so regularly?

You're really thinking of finance. Pure banking is solved long ago.

That's ok, but based on your example, how would a human apply an algorithm that it can't determine why it worked?

We're not talking about "Not Hotdog" here. Application of a model in a real-world at-scale scenario is a lot more than running inference and walking away.

At a bank credit decisions are evaluated by humans, frequently and often. These reviews are conducted in the forms of sampling audits, control processes, and other scenarios that involve internal bank employees and external regulators. In each case, humans will inspect the details of what occurred. This would be impossible with any type black-box model (SSL, deep NN, etc.).

I'm not sure I understand your comment.

Bank credits are auditable, because there are some precise algorithm around them. Humans created the algorithm specifically so that it is interpretable. Then they must apply it the same way to everybody. They can't just say "oh, I think this person is more likely to actually reimburse, I play golf with him and I trust him !"

This is a specific case where no black box will ever be used (at least I hope).

There's incentive to use complex black boxes everywhere, if more profitable.

Of course, and my point is that it's perfectly fine, and even desirable, as better models lead to better outcomes.

Of course, for cases where explainability is needed, either required by law (such as banking), or by common sense, then black boxes will not be deployed.

Self-supervised learning is very promising. But it is strange to me that so much computer vision research has no mention of the 3d structure of objects.

That's why I like things like this https://arxiv.org/abs/2102.12627 "How to represent part-whole hierarchies in a neural network" by Hinton.

Yeah, humans don’t learn from still pictures, we see things moving through time and space and rotating. This is likely providing us with essentially a lot of “self supervised” data. For instance https://www.ted.com/talks/pawan_sinha_how_brains_learn_to_se... starting at 8:11

Great video. It's interesting to see how far semantic segmentation has progressed since 2009.

Not directly related to the SSL part, but I have a question about how the work on AI is going. - Why are we even trying to create an AGI when we have no idea how it will turn out? - AI in day to day functions can be useful and I'm all for that but what is the point of even trying to give the AI self-awareness?

So it can handle the strategy of making sure no one else makes one which is more harmful. :P

But as a serious answer: of course, there is no major economic use of trying to make sure it has an internal subjective experience, but that’s not what people are aiming for when referring to AGI. The goal is that it be able to accomplish general tasks and goals. Like, what can you do with it? Everything you can do at all. That’s what people are aiming at. What tasks of reasoning and planning could a person do for you? An AGI would, basically by definition, be able to do those same kinds of things. (Of course, something could be an AGI but not as intelligent as a typical human, so long as it could still reason about all the same kinds of things. But at that point it would presumably just be a question of scaling things up.)

AGI definitions are incredibly fuzzy and agency reliably seems to be confused with intelligence.

The fundamental problem is whether AGI is outer-directed or inner-directed - i.e. whether it sets its own goals, or whether you tell it what to do and it improvises a solution.

AGI is most useful when it's outer-directed with limited agency but some improvisational autonomy. You can give that kind of AGI specific problems and it will solve them in useful but unexpected ways. Then it will stop.

AGI is most dangerous when it's inner-directed with full independent agency. Not only will humans have no control over it, there's a good chance humans won't even understand what it's aiming for.

Agency is almost entirely unrelated to symbolic intelligence. You can have agency with very limited intelligence - most animals manage this - and no agency with very high symbolic intelligence.

This is not a Boolean. But there will be a cutoff beyond which inner-directed behaviour predominates, initially driven by programmed "curiosity", leading to unpredictable consequences.

Can you elaborate on the outer-directed vs inner-directed distinction (or link to something else which does, if that would be more convenient)?

I'm not quite sure what you mean by "sets its own goals" (for "inner-directed"). I assume you don't mean "modifies its own goals" (as, why would that help further its current goals?), but I'm not sure what it would mean. Maybe you mean like, if it acquires goals and preferences in the way that humans do, with shifting likes and dislikes that aren't consistent across time? Yes, that would certainly be quite dangerous (unless the AGI was via like, just emulating a human's mind, which might not be so dangerous, in which case only somewhat dangerous).

I believe I see your point in the safety features in having it receive a specific short term task, achieve the task, and then stop. Once it stops, it isn't doing things anymore, and therefore isn't causing more problems. But it seems like it might be difficult to define that precisely? Like, suppose it takes some actions for a period of time, but complicated and anticipated-by-it-but-not-by-us consequences of its actions continue substantially after it has stopped?

Does not anyone worry about this whole human level AI Pandora’s box trap? Why not work on nukes themselves, sounds safer to me

Given that we all haven’t been obliterated by nukes, it does seem your argument seems to argue against your own premise. We will probably be fine even if we inexplicably create superior intelligence. Besides? Who’s to not say that the best course of action an infinitely intelligent being would choose is to just switch itself off given the futility of everything?

Don't worry. A sufficiently powerful AGI won't destroy the universe to satisfy its own goals; it'll just build a holodeck (by tampering with its sensors) so that it can gain +infinity pleasure right away.

Unlike software, which can be mathematically airtight, there's no reason to assume that physical devices built by humans are unexploitable in the face of the massive intelligence of the AGI. So the AGI will just ask itself which is easier: delicately performing surgery on its own sensors, or building tons of nukes.

Honestly, that people are even worried about AGI doomsday shows to me the power of narrative. Everybody has heard of SkyNet and the sorcerer's apprentice, nobody has heard of the AGI who outsmarted itself by creating digital porn for its objective function. Therefore, through the miracle of the availability heuristic, AGI doomsday it is.

i'm not as scared of real AGI as I am of some pseudo-AGI convincing enough to fool everyone into following what it comes up with and ascribing it magical powers - we have enough problems with personality cults and herd mentality as it is..

Looking for more visuals and less verbal

These are interesting epistemological questions. Is there a compelling reason to label a certain neural architecture as a "siamese network"? I'm guessing that this is a reference to conjoined twins, a biological phenomenon with no connection to the country previously known as Siam (now Thailand).

If you cut some words out of a newspaper and a person writes words in those blanks then what has happened? If we cut out parts of a painting and a person paints in those blanks then what has happened? How is the person who wrote in the blanks or the person who painted in the blanks changed by writing or painting in the blanks? How are they changed when someone responds to what they've written or painted? How did the original writer or painter come to write and paint what was later blanked out?

Here are some words that I read but did not grasp firmly: learn, data, task, train, intelligence, general, model, skill, label, language, understanding, reality, observation, predictive, objects, concepts, act, hypotheses, knowledge, 'common sense', 'dark matter', 'artificial intelligence', teaching, classify, supervision, autonomous, 'self-supervised learning', recognize, patterns, representations, processing, systems, pretrained, vision, real-world, helpful, promising, 'energy-based models', prediction, uncertainty, 'joint embedding methods', 'latent-variable architectures', reasoning, 'predictive learning', 'supervisory signals', signals, structure, unobserved, property, input, 'co-occuring modalities', 'unsupervised learning', feedback, reinfrocement, 'downstream tasks', meaning, syntactic, word, associate, probability, vocabulary, 'convolutional network', network, 'prediction uncertainty', 'predicting missing words', computing, softmax layer, probability distribution, energy, incompatible, computer vision, and so on.

One could make a similarly long list of poorly understood concepts in psychology.

I've found that plucking words from paragraphs has, on occasion, brought them more firmly under my control-- most of them remain mysterious to me when they pop up in papers on programming or psychology.

Rather than list "ideas" or "concepts" I try to make a record of what appeared and the response I had to its appearance.

How are we to solve difficult problems if we stop mentioning them or their difficulty?

If the person who filled in the blanks already had a good grasp of the language, they probably didn't learn much. However, fill-in-the-blank exercises can be a good language learning tool for children as well as adults learning a foreign language.

There is something special about that "lock and key" moment when you feel that satisfying "click" as your response opens the next opportunity.

Escape rooms for language models :)

(Context https://en.m.wikipedia.org/wiki/Escape_room )

The person isn’t changed, just like the weights aren’t changed. You’re missing the training step - after the person fills in the blank, then we tell them what was supposed to go in the blank. If I had you try to paint the the rest of the face of a portrait, you may do a poor job, but then if I showed you exactly what it was supposed to look like, you can see what you did wrong, and next time you will draw it better.

> How is it that humans can learn to drive a car in about 20 hours of practice with very little supervision, while fully autonomous driving still eludes our best AI systems trained with thousands of hours of data from human drivers?

Wow, are these so-called ai scientists really that daft?

Sure on paper we drive 20-40 hours "practicing" then hit the roads, but we've been back-seat driving and driving via video games or TV since the day we're born.

While a 2-year old doesn't know the intricacies of driving my 3-year old can definitely yell hey dad the lights red slow down.

Full-immersive life experience. Perhaps ai's need to be put into a real-world "birth" simulation (perhaps we already are this experiment) to learn as they grow.

The problem I see is the narrowness, if you're training on a narrow subset then you're gonna get narrow results. I don't know the best way of doing it but you need to start thinking of an ai's "brain" like that of a child's and how it absorbs things - the human brain is remarkable sure, but I don't doubt it's duplicatable in silicon.

I think you misunderstood. That’s exactly the point they are making.

I think you miss the point, it's not strictly about "it take 20 hour for an child to drive a car" and more of it can do this without external help. From your example, It's about how the child knows to stop at red light when he was never told that you have to stop at red light.

Reply on news.ycombinator.com