MMC4: An open, billion-scale corpus of images interleaved with text
github.com
github.com
This training corpus mmc4 is the one used by OpenFlamingo (https://laion.ai/blog/open-flamingo/).
Something caught my eye when I read the pdf. They are using GPT-4 to name their word clusters: "Topic names are generated by GPT-4 conditioned on the top 20 words for each topic, prompted by a request for a short 1-2 word summary."
I feel like LLMs will be used increasingly for this purpose; whenever someone has to make a mostly arbitrary decision like the kind that would make 'bike shedding arguments', they can delegate it to the LLM. The point isn't that the LLM is necessarily better or objective, but rather that it's a kind of 'schelling point' for agreeing on things that don't matter.
For example I could imagine a super annoying peer reviewer (or even co-author) ask why you used the name 'celebrations' as a topic name associated to the set of words 'fun, wedding, beautiful, christmas, happy, card, birthday, gift, blog, perfect' like in the pdf, and why not use some other word. Instead of having some meaningless back and forth 'bike shedding' discussion with the reviewer over email, you can just say you used GPT-4 and move on to more important things.
How can companies such as OpenAI use that data when no licenses can be identified for most web pages or the associated images? In other words, the user/owner of the content has not agreed for it to be used in these ways.
What's people's experiences with training models using such data?
Is there any way to identify if my photos appear in common crawl or are used in such datasets?
Analysing an image and adjusting weights in relation to that image is not making a copy. The images aren’t being copied into the model.
You could argue that downloading the images in the first place to make that analysis constitutes copying, but these kinds of incidental copying aren’t normally considered within the bounds of copyright. If they were, you’d be committing copyright infringement every time you surfed the web.
AI is a big brain which consumes the web almost like humans do. It "sees" the pictures on the web when it adds their characteristics to its associative memory. The AI then generates images. But unless it comes up with a clear copy of somebody else's work, it is not copyright infringement.
I am not a lawyer so take this with a grain of salt.
Except where they kind of are.
Where it gets fiddly is if the model is capable of spitting out close reproductions of images from its training set. That's been demonstrated on some models. I don't have a particularly good intuition as to which way this would go, but I can see someone making an argument that copying the model is therefore copying all the training set images it could be convinced to make a reproduction of. That's unlikely to be everything in the training set, but by the same token I don't know how you could determine how much of the training set can be reproduced that way.
Am I not allowed to do this either?
I download an image and save it and than use it. Either as inspiration in a collage or as inspiration when drawing something new.
I don't think there have been a lot of people out there who really invented something uniquely new for a while.