Machine Unlearning Challenge | Hacker News Reader

Machine Unlearning Challenge

ai.googleblog.com

172 pointsAnon8431 comments

Joel: "Is there any risk of brain damage?"

Dr. Mierzwiak: "Well, technically speaking, the operation is brain damage, but it's on a par with a night of heavy drinking. Nothing you'll miss."

- Eternal Sunshine of the Spotless Mind (2004)

Lord Mayor of Cologne: "You've damaged your brain, Universe, but no more than a week of binge drinking or five minutes on a cell phone."

- Futurama, Parasites Lost (2001)

Homer Simpson: Alright brain, you don't like me and I don't like you. But let's just do this and I can get back to killing you with beer.

- Simpsons, The Front (1992)

None of this is true for those who abstain from Alcohol :)

Did you miss the clause 'on a par with'?

…yes

They say privacy, but this sounds like a request to make censorship more censor friendly.

Edit: People seem to disagree, but is this not a tool for wiping ideas from your training data? Meaning they want to train on Wikipedia and remove concepts (per locale probably) easy as pie. This is Google outsourcing compliance with jurisdictions that require thought policing.

The obvious use case is to comply with things like copyright and data protection laws (e.g. GDPR) without having to discard all models that used data unlawfully in training.

Even though they talk about removing biases from the model, I doubt this would make the model unlearn whole ideas/data patterns without severely damaging performance, which they clearly don't want.

While this post seems aimed more at compliance or sensitive data issues, “unlearning” aka forgetting itself may be essential for better or more human like AI agents. You are as much defined by what you have forgotten as what you have learned.

I've heard than some mental illness (maybe schizophrenia) is theorized to be in part due to an inability to forget things. Certainly normal human existence involves preferential remembering and forgetting. For example not dwelling on pain or failure etc.

Gradient descent obviously has none of that, it has not feelings or goals, and so there would be no preferential remembering or forgetting other than to do better next word prediction or rlhf or whatever. So it would be interesting to think about what a model should remember or forget in order to align it with our goals (because it doesn't have any of it's own).

Also, don't volunteer for Google. They have lots of money, they can pay for this stuff and if you know how to do it you have lots of options.

Yeah, the prize for doing the best on this challenge should be a piece of the company.

> I've heard than some mental illness (maybe schizophrenia) is theorized to be in part due to an inability to forget things.

My totally amateur pet theory is that paranoia is threat pattern recognition gone bonkers.

Actually, most of what we do with brains is pattern recognition, and if there isn't enough good input they will make shit up.

> My totally amateur pet theory is that paranoia is threat pattern recognition gone bonkers.

Supposedly OCD is your brain’s cause-effect loop being too potent. As in, you have a random fear I.e. “stove is on, fire will burn down house,” you go to check the stove, and the act of checking (regardless of its being on) creates the feeling that you saved your house from burning down, so now you feel compelled to check every time. Paraphrased from the Huberman podcast.

I have OCD. This is basically accurate in that it's an anxiety disorder that affects your ability to judge whether a worst case scenario is likely or not, but the thoughts are more like "What if I drive across the median and kill a family of 5 in a minivan" or "What if I pull my genitals out in this meeting and try to fuck my boss". The compulsions are usually not as logically coherent as needing to check if the stove is on or the house will burn down (though there are people who experience ocd like this). Usually my compulsions happen as a way to try and stop thinking those thoughts, like pulling over and playing internet chess for half an hour. The problem is that the compulsions steal a lot of time and don't typically address the underlying obsessive thought.

In 1999 I dated a girl whose father was a military doctor. He was the only person whom I ever knew to have an almost total recall. Like, you asked him what happened on this particular day five years before, and he told you exactly what. If there were records (like photos), they would match his words.

He was also quite unassuming and very "normal". No trace of mental dysbalance.

it is certainly better for all AI agents, not just human like systems, because it is almost the definition of intelligence. Forgetting the irrelevant stuff and having a system that does as much as possible with as little information as possible is what learning is, compression in other terms.

I always though it funny that in short term, LMMs are (or can be) much better at forgetting than humans can. If a person tells you, "I'm going to forget you ever said that", you know they won't actually forget. An LMM, on the other hand, can just drop the context, and they're literally going to forget it ever happened. They already train chatbot agents with a special "end the conversation" token, it would be easy to train them with a "drop the context" token too.

An interesting idea. My first thought was to say “just use a differentially private learning algorithm,” but DP requires a notion of closeness, and that’s not trivial for, e.g., image data. Images close in an abstract sense, like Euclidean distance, may not be close in a task-relevant sense. You may not know what a good, task-relevant metric for employing DP without performing some kind of learning in the first place.

Who knows… unlearning might turn out be good as a “sleep” phase while learning too. Sort of a regularisation. Could be done after each epoch.

Please don’t do free work for google. If you have a solution for this then you have lots of options.

Is there like a cash prize for solving a major problem with one of Google products? It seems disingenuous for Google to frame this as some academic challenge when it's clearly for Bard, Google's commercial product.

As with many ML-phrases, the use of 'unlearning' frames data modelling as a false dichotomy between 'learnable' and 'forgettable' data. Whereas humans are able to forget over time, it would be quite disturbing to block some memories from access entirely, how traumatic they might be (and quite apparent among people suffering memory loss).

While from a privacy perspective, not all 'data points' (i.e. memories) need be shared with the populace at large, to the individual they do constitute affective buildings blocks to deal with previous experiences. Similarly, the threshold for sharing (private) experiences depends on the communicative context one's in and how comfortable one feels sharing them.

Might similar mechanisms provide large models the ability to retain an internal recollection of 'traumatic' or 'problematic' data and become more attuned to the context its communicating in instead? Leaving blank spots or blocking out experience completely sets the stage for 'disturbing' a model's memory after the fact. While I don't want to draw a comparison between current NN architectures and organic ones, it is worth questioning our framing of currently emerging methods, as the incipient terminology can imbue how later practioners implement and think about these mechanisms.

It's just statistics guys. You either put the data in the training set or you don't.

Well, isn't this challenge about altering a model after-the-fact? I.e. we are stuck with this data in the training set and must figure out a way to clean up the model

Yeah you have to retrain it lol. Sorry about your $100 million model, shouldn't have used all that data indiscriminately.

I'll drop my idea here since I won't be participating. Trade disk for privacy. Basically you keep all your initialized weights, gradient updates from the training run and an index of what samples appeared in what batches. Then when you need to delete a sample, you find all the batches containing the target, reconstitute batches without that sample, and save those updated batches. You then take the initial weights and apply all the gradients that didn't come from contaminated batches. Finally you run a small additional bit of training with the cleaned batches.

This idea doesn't fully remove the influence of the target data (any previously saved gradient update from after a contaminated batch contains some information about the state of the network prior to update) but it may be a sufficient and efficient way to quickly reconstitute a network with far less influence from the problematic data.

Just an idea and I haven't tried it, so maybe it's bunk, but there you are!

Did you do any estimate of how much storage is required?

On the face of it, I would expect the gradients to take about as much space as the weights. So you’d be checkpointing your network at every batch, in effect.

They say privacy, but this is specifically an anti-privacy tool that can be used by anyone holding the before and after of the unlearning.

This is an interesting concept. Is there something like an MNIST for this, to make it more concrete?

Yes. Like all NeurIPS competition challenges, there is a starting kit which provides a dataset and metrics, there is also a grader on Kaggle.

Deep forgetting

Reply on news.ycombinator.com