Hacker News Reader

Comment by hlfshell | Hacker News Reader

It started by accident, with the original llama weights being leaked by two separate employees. They've since embraced opening the weights, which I'm all for.

As for why? I have a theory: Meta is not in a position to capitalize upon the model itself. Yes, they can use it internally, and maybe their competitors can copy it to - but there are no real competitors to Facebook or Instagram that can benefit from it enough to make it a differentiating facet.

Thus, releasing stuff for open source does two things:

1) Make them more attractive to research talent (Apple famously recently started to publish research because their traditional secrecy was causing issues with hiring top talent) and...

2) Continues to undermine the ability to make $$$ off of model alone, driving it towards being a commodity rather than the long term profit engine for other companies.

View on HN

Wrong. FAIR has been open sourcing ML models and source code for the last 10+ years. It did not start with llama. Also, llama was not leaked by employees, but people in the broader community, who the weights were shared with.

For example:

Faster R-CNN - state of the art image segmentation, released in 2017.

FastText - text embedding models, 2016.

FAISS - vector DB, 2018.

https://github.com/orgs/facebookresearch/repositories has over 1,000 repos.

I was speaking about LLM weights specifically and llama, not all models and work at FAIR.

Per your links, it's clear that FAIR does have a good history of open source work.

This is very true. But we should also mention that Facebook sadly have been and are on a negative trajectory of openness. As someone working closely with them, there was a culture of nearly complete openness in the early years of their existence. Research was promptly shared in its entirety and licensing was compliant with open science. However, as the "AI boom" has grown, there is an increasing internal culture of holding parts of research back (my understanding is that this pressure comes from the C-suite). Licensing that previously was open source compliant, has had non-commercial clauses added more and more frequently and even non-standard complex agreements as we have seen for LLaMA 2 and 3. This is sad and the culture of openness is ultimately at risk as they become more and more like OpenAI, Google DeepMind, etc.

As I frequently point out, Facebook are free to decide on their own culture as they see fit and I am not entitled to their work. But it saddens me that they believe that compromising on their initial ideals is the way forward, rather than sticking to them through thick and thin. This, ultimately, makes it more and more difficult for me as an academic that believe in these ideals to work with them.

> It started by accident, with the original llama weights being leaked by two separate employees.

This is not true. Meta Fair has been built on openness from day 1. We published many papers and open-source d many repositories to reproduce the work

In short: "Commoditize the complement."

> weights being leaked

You can hardly call that "leak" when they basically were sending the weights to thousands of people who applied for access. It is not that they kept them secret.

Reply on news.ycombinator.com