Hacker News Reader

Comment by oofbey | Hacker News Reader

One thing I find really interesting about the DALL-E-2 is that the popular blog name ("DALL-E-2") never shows up in either of the research papers that describe it. The paper commonly referred to as DALL-E-2 calls its own algorithm "unCLIP". UnCLIP is _heavily_ based on a paper from a few months earlier called GLIDE - in fact you can't really understand the unCLIP paper by reading it without first reading the GLIDE paper.

I suspect what's going on is that OpenAI has decoupled their PR activities from their science activities. They told the researchers to publish papers when they're ready, and then the PR apparatus decides when one is good enough to be crowned "DALL-E-2" and writes a blog post about it.

View on HN

This is only surprising if you're not familiar with product launches that result from R&D. In this case DALL-E 2 is the consumer-facing name, unCLIP is the name used during research and in this case for publication. OpenAI may also have a further internal codename that they used for the project. Currently DALL-E 2 access is limited but there are lots of reasons to believe that OpenAI will try to productize Dall-E 2 as an API. If you're selling a product, you need a product name.

I honestly hope they don’t productize it until they can build a dataset of images that have been explicitly licensed for remixes without attribution. Having proven that the system works, they could now justify the effort to build a very large public dataset of CC0 images. For example they could encourage Twitter to add image licensing options and use the alt text of images to form the image description. They could then encourage twitter to encourage the use of licensing and alt text, and foster a culture of open sharing. There are already 500 million images in Creative Commons, hopefully many with an appropriate license for reuse without attribution.

Otherwise what I am hearing is that artists online are upset their work has been incorporated in to DALL-E without their consent. A system that rips off artists and then puts them out of work would make OpenAI and the AI community look bad.

Instead we could move forward with better licensing and alt text features for major platforms and draw in an open community rather than taking what has not been licensed for this kind of use.

Same with all these big models. GPT-3 and codex benefit from all the text and code that has been made publicly visible without giving back. At least e.g. imagenet classifiers had to take the effort to label the images they scraped.

IP laws need to catch up, so that ML models that use unlicensed public content have to give back in some way.

I disagree. If it’s on the web it should be fair game. These models are not directly cutting and pasting, it’s my opinion that they’re learning some higher level features and attributes. Admittedly, not everyone shares that opinion. Humans don’t exclusively learn from licensed content and I don’t see why our learning algorithms should be restricted to that.

It seems to me there is a gap in robot rights vs human rights:

Google has scanned in much of the world's library collections - the search agent is allowed to look at all of it, but I am only allowed to see samples. How is it Google is allowed to keep a copy on hand, but is not allowed to show me?

And now OpenAI has done the work of curating hundreds of gigabytes of publically viewable images (copied without regard for copyright, since of course that's how the internet works, but they have made a copy of each of these images for their training set) -- it seems strange to me that if I want to ask DALL-E, "what image in your training set is the nearest match to this output", copyright law prevents it from showing me (perhaps it could at least return the URL it recorded, and I can hope that Archive.org has taken on the risk of violating copyright on my behalf)

In any case, intellectual property laws have created a world where the robots are compelled to withhold information from us mere mortals, with the excuse that they can't make one more copy for us.

Is it really that surprising, though? The training data is more like 'inspriation' for human artists. Certainly, there are cases in music where sampling or melodies are a little too close to the original and spark lawsuits, but largely, very similar derivative works abound.

Nothing about creating a derivative work, intentional or otherwise, would allow you to use the original in a way copyright protects. Even Fair Use, which is fairly permissive, doesn't allow for blatant disregard of the original copyright.

That's an interesting thought experiment. Every human btain contains illegal copies of music and images. If you hum or sing or even think about an IP, in the future you will have to pay a license fee to a corporation.

It’s already illegal for you to sing a copyrighted song in front of a group of people. If you hear a Taylor Swift song and memorize it, then later sing it in front of a sufficiently large group of people I believe that is illegal.

Now I don’t actually think intellectual property restrictions are good. I think they are quite harmful. But as long as they exist I don’t like the idea of big companies trampling over the IP rights of small creators. People currently depends on those rights. One of the reasons I want to see a large public dataset of explicitly licensed images is that if OpenAI is required to do this, they would have to spearhead a new culture of openness to get enough data. Imagine if they encouraged twitter to add a license option and then simultaneously encouraged users to add alt text and to license images with an open license! This would be a boon to people with sight disability who require alt text and it would expand a visual commons of openly licensed data. Instead we have one company making a big dataset full of copyrighted images (sharing something online does not waive copyright). I think a model based on explicit consent is healthier for everyone in society.

Why as an artist should I assist in my own demise? If you are coming gunning for my way of life and skill I've spent half a lifetime developing, my very purpose, don't expect me to just roll over and help you.

We still own the rights to all our material. I don't see why we can't en mass expressly forbid the use of any of our creative works in machine learning training sets. That's a specific and measurable point where our material is inserted in to a machine without prior permission and with malicious intent towards us.

DALL-E is mentioned in several places in the paper.

DALL-E 2 specifically is on page 18 and the system card: https://github.com/openai/dalle-2-preview/blob/main/system-c...

DALL-E 2 = the stack of unCLIP and the image generator.

I noticed that as well! It confused me a bit at first. They say that their "image generation stack" is referred to as unCLIP, and I was trying to figure out how it's distinct from DALL-E 2 at first!

My only guess would be that unCLIP is the end-to-end image generation model, but if the model is used for manipulation, interpolation, or variations, then it is referred to as DALL-E 2. So unCLIP is a subset of DALL-E 2.

Maybe it's because there is more features on the openai websites? for exemple with GPT, you get different models, different templates, a playground, an api etc...

Unrelated: I've noticed the common use of underscores for emphasis in HN comments. Why use that when italics are supported via asterisks? Like this?

Markdown syntax is that a single underscore around words _like this_ renders it italicized.

Sure, but HN doesn't. I'm trying to understand why I see so many comments using underscores when only asterisks work.

I think it's a combination of Markdown being completely readable in plain text especially if you're familiar with the syntax and even if you're not. Similarly, I see a lot of people using TeX-style mathematics, it's not particularly readable but "The quadratic formula is $x=\frac{-b\pm\sqrt{b^2-4ac}}{2a}$" is a decent way of representing the formula to people fluent in LaTeX even in plaintext conditions. I suppose there's also likely to be a bit of muscle memory where people accustomed to typing in Github/Stack Overflow/Reddit markdown use it on other systems, and even if they see it's not supported it's good enough to not need editing.

I don't think it's particularly worthwhile to learn a new comment format (one that's not even linked or described in the comment editor, for that matter) for every site.

Aren't asterisks standard in Markdown syntax too?

this was also a common convention during the usenet news era

_Underscores_ still work, even if they don't get converted into <u></u>, they convey the meaning just fine.

Similar to how people use ">" to indicate quotes, even if it doesn't get special treatment by the editor.

Interesting. It doesn't work for me; I have to remember that underscores parse to italics and then "apply it in my head", rather than just reading italics, which gives them a distinctly different quality. For example, I just heard your response in my head as "Underscores still work", which is different from "Underscores still work." They also don't pop out as much when scanning text, as italics do visually. I have to spot the underscore before and after the word, and it really doesn't work if you try to emphasize more than one word. At least with the ">", it's at the beginning of the line and in a separate paragraph.

HN's markup is idiosyncratic but similar enough to markdown that it's hard for occasional commenters to remember the details. It's also minimal enough that users are already used to parsing extra-syntactic markup visually.

Yeah. HN should just switch to markdown. ;)

I don't get it, isn't this how literally every product launch works.

I don't know why you're being voted down, internal/research names are often way weirder and decided on ad-hoc by the researchers themselves and then a good PM comes in when product-ionizing and part of this is deciding on a catchy name for public use.

The phrasing "I don't get it" is fairly rude - it implies the post is obvious to the point of not being worth mentioning. However obvious this might seem to somebody, I would point out that turning AI research papers turn into products is hardly commonplace.

This behavior is not exclusive to OpenAI. NVIDIA did this too. Originally StyleGAN3 was published under the name "Alias-free GAN" and the paper itself uses that terminology.

unCLIP as in “this algorithm is NOT going to be told to make paper clips resulting in all the mass in the solar system converted into paper clips”?

Reply on news.ycombinator.com