Hacker News Reader

Comment by tasdfqwer0897 | Hacker News Reader

Hey I work at Adept and helped make this! Happy to answer questions. The thing I think is especially neat/notable is how simple you can make the model architecture while still getting good performance. I expect we'll continue to see bits of these models get deleted in the next few years

Note that you can get the model weights on HuggingFace here: https://huggingface.co/adept/fuyu-8b

View on HN

First off, absolutely incredible work, congrats and thank you.

Secondly, do you anticipate Fuyu being made available for commercial access or will it remain NC?

What’s the situation with the license? Your blog post says you are open sourcing it, but it’s currently only available under a non-commercial license instead. Is an open source release forthcoming?

You can download the weights on Hugginface.

I believe the copyright on AI model weights in the US is not fully established, but so far it has been held that a list of numbers can not be copyrighted, so likely the same applies to model weights. Note that you don't have to enter into an agreement with Adept to use the model.

Alternatively, use and download the weights in Japan that has explicitly no copyright on AI models.

> a list of numbers can not be copyrighted

Any digital object can be represented as a list of numbers (this is precisely the origin of the term digital). Since there is clearly precedent for copyrighted digital objects (media, software, etc), reducing something to "a list of numbers" is not a useful distinction in regard to copyright law.

IANAL but as far as I remember, you can't copyright a list of objective facts, for example a phone book containing a list of phone numbers.

Model weights are clearly not in that category. Happy to be corrected if I misremember.

Model weight are akin to markov chains and compressed data. They are direct representations of the data they where created from in the same way that markov chains are created from hidden markov chains and Zipped files are created from files.

Zipping a file does not grant the copyright protection of the zipped output beyond the copyright of the original file.

Moreover the American federal registrar has officially stated that AI generated artifacts are not eligible for copyright https://www.federalregister.gov/documents/2023/03/16/2023-05....

If you take some copyrighted data, a set of books, for example. And count words in these books and then plot a distribution of top 100 word frequencies. The copyright for that new image would belong to you.

Copyright in the specific image sure, but not the graph itself. Someone else could do the same thing and make their own graph image.

Exactly. Data is not covered by American copyright and artifacts generated by LLM and diffusion tools are not covered by copyright protection unless there was human involvement and humans are transparent about how they participated in the creation of the artifacts.

For now there is a lot of human involvement. You pretty much need a team of engineers or an equivalent to get anything besides minor fine tuning done. And there is usually human labor involved at labeling, feedback and evaluation stages.

The issue circles back to their needing to be transparent about how they did the work.

When it comes to intellectual property there are two methods of protecting it: either you can keep it a trade secret and only use it in house (the secret sauce approach) or you keep things out in the open and seek copyright or patent or trademark protection. You can't have it both ways and even more so with AI co-created artifacts. If they are transparent about all the steps involved and what the humans did then they can seek protection for the human created parts. This also allows others to then replicate these steps and to create similar artifacts.

It sounds like they and many other "AI" teams want patent protection without having to register for it. These teams are trying to write their own licenses to rights they do not have.

I highly doubt that any of this will hold up infront of a court. For intellectual property not just the result is important but also the creation process, and there is enough work going into the data science here

Yeah... in the blog post, they do explicitly mention "cc-by-nc", which I find disappointing.

Anything that Adept is "excited to see what the community builds on top of it" would only serve Adept and no one else! What incentive does the community have to build on top of Fuyu, when the community can't benefit from its own work? If Adept wants to benefit from word-of-mouth discussion of their models and from community contributions that make those models work better, as has happened dramatically with Llama 2, then they need to give the community the opportunity to benefit too.

Also weird: if you look at the tags on Hugging Face, you'll see it is listed as "cc". This comes from the README[0] metadata. "cc" is not really a license.

[0]: https://huggingface.co/adept/fuyu-8b/blob/main/README.md?cod...

It's open source by their definition, that is source available (open). Everyone always thinks the term open source is protected in any way while the entity that has established the commercial usage aspect is the Open Source Foundation. And noone is forced to abide by their ideology

FOSS meets the commercial usage requirement much better. Otherwise the term FOSS would be redundant.

Hey! Awesome work. It seems like in theory this encoding scheme should enable the a model like this to generate images as well, by outputting image tokens, is that right?

Thank you for the release!

What can you tell us about this:

> Our internal models (based on Fuyu) have extra capabilities related to our product. In particular,

> 1. They can reliably perform OCR on high-resolution images

> 2. They can do fine-grained localization of text and UI elements within those images

> 3. They can answer questions about images of UIs

Is this just a matter of additional fine tuning, or are there architectural differences?

Even with experiments with just adding additional fine-tuning, we've seen models gain these capabilities!

Neat idea! Are the batches encoded as tokens into the input sequence? This is something I really like about the multi-modal PALM papers since it enables the multi-modal tokens to be referenced.

Image patches are projected directly into an embedding that goes into the decoder Transformer. The same thing could be done for audio.

Hi, Will it work for html/APP UI screenshots, Have been trained using UI screenshots ? Thank you

Do you offer paid API access to larger models?

can you guys launch a web gradio demo until the transformers PR gets approved? i'd like to play around with the model

Reply on news.ycombinator.com