EleutherAI announces it has become a non-profit
blog.eleuther.ai
blog.eleuther.ai
What we do need is more weights being released in the public domain (hard to find even on Huggingface), easier ways to train models locally, better pruned models for embedded device inference (e.g. running on a Jetson Nano), easier ways to fine-tune for specific contexts, and so on. My big gripe, and for obvious reasons, is that we need to step away from cloud-based inference, and it doesn't seem like anyone's working on that.
The profit motive pushes organizations to keep their work secret (which is what has happened with OpenAI).
I think there's still something of a financial disincentive to promote selfhost over cloud capabilities for just about every party involved except the selfhoster. NVidia loses out if they're selling a few Nano or Orin platforms relative to the much pricier datacenter cards sold in huge lots. The cloud hosters are making up for the cost of hosting in terms of end-user pricing, snarfing all that lovely customer data and likely selling analytics, and probably other measures I can't understand yet. And the large companies that fund research and initial model development want to know what's being done with their models so as to gain any possible competitive advantage. They can't necesarily guarentee that intel from a self-hoster. Almost nobody is willing to spend the time explicitly to make it easier for the individual dev at a small lab or at home to do this because that's essentially a donation and not a business expense that might yeild obvious returns.
I think there are steps being taken in this direction (check out [1] and [2] for interesting lightweight transpile / ad-hoc training projects) but there is a lack of centralized community for these constrained problems.
[1] https://github.com/emlearn/emlearn [2] https://github.com/BayesWitnesses/m2cgen
Also: https://twitter.com/elonmusk/status/1626516035863212034
Edit: Plus it's still the case that OpenAI aren't putting profit above all else. E.g. they just released the ChatGPT API and it's 10x cheaper per token than GPT3! And their Charter does count for something, it's binding on them.
Not remotely an expert, this is just my basic understanding. "Nonprofit" is a pretty abused term.
Overhauling the legal definition of non-profit to require the highest-paid employees to make no more than ~10X as much as the lowest-paid employees would make a lot of sense.
I think you are confusing non-profit with charity perhaps?
As one example, the CEO of Kaiser was the highest paid nonprofit CEO at ~$18m[0] in 2021. That seems egregious until you realize Kaiser had $93b in revenue in 2021[1].
As one example from healthcare, the CEO of Moderna was paid $18m in 2021[2]. Moderna reported $18b in revenue in 2021[3].
You see this over and over in discussions about nonprofits, charities, etc. In most cases the executive leadership team is running a huge organization - in the case of Kaiser nearly $100b in revenue and 300k employees. Competent leadership at that scale is expensive.
In an ideal world this wouldn't be the case and individuals with these qualifications would be willing to take even more significant pay cuts for the "greater good" of whatever the mission of the nonprofit it. Per usual that's not the world we live in.
Speaking personally, I'd give pause to working with, donating to, etc an organization doing $100b in revenue with an executive team making 1/20 (or whatever) they would make elsewhere. In some limited cases for those who have "made their money" it would be a great thing. For anyone else I would assume they're either taking a HUGE pay cut for the cause (great thing, but unlikely) or completely incompetent, embezzling, etc.
[0] - https://www.erieri.com/blog/post/top-10-highest-paid-ceos-at...
[1] - https://about.kaiserpermanente.org/news/kaiser-foundation-he...
[2] - https://thehill.com/business-a-lobbying/business-lobbying/34...
[3] - https://investors.modernatx.com/news/news-details/2022/Moder....
Very easy to work around by outsourcing most operations. If you really want to prevent high salaries there you'd need to link it to some more fixed measure, such as a multiple of median salary.
A privately owned for-profit company can do whatever it wants, good or evil. OpenAI is not publicly-owned.
Did you donate money to OpenAI?
A nonprofit is an organization that uses its income and profits for the organization's main goal that supports the mission. On the other hand, a charity is a type of nonprofit that engages in activities aimed at improving lives in the communities.
NFL was a non-profit trade association until ignorant whiners made it bad PR and they changed it to shut up the whiners.
That's not a contradiction. A non-profit can be profitable. The profit just needs to be reinvested back into the mission of the organization.
Non-profit doesn’t mean the organization isn’t supposed to have surplus revenue, it means it doesn’t exist to return profits (e.g., via a claim on the assets of the company) to stockholders or some other beneficiary with a claim on them.
Isn't Ikea technically a non-profit? through some weird structuring?
Can you elaborate on this “obsession”? Aren’t a lot of AI things made/implemented by for-profit companies? Google, Meta, Midjourney, etc?
There's no shortage of non-profits that exist to pay saleries. As an example, growing up in the bible-belt, and I can confirm that religion can be good business.
We also developed VQGAN-CLIP and CLIP-Guided Diffusion, techniques for doing text-to-image synthesis that don't require training and can easily be run locally for inference.
They suck in public data, create models, let them loose, let hyped people improve them and profit off of data and people's free work.
IOW, a mechanical Turk which doesn't pay.
But I’m confused by the anti non-profit vibes in this comment section. We aren’t saying that becoming a non-profit makes us ethical people, that would be a silly argument. But people do realize that the alternative would be to become a for-profit entity right?
We’re still the same community-driven open collaborative research lab we’ve always been. But incorporating allows us to do things like hire full time staff, enter organizationally binding legal agreements, and protect our members. Between the options of becoming a for-profit and a non-profit, the later seems clearly better suited for our goals.
A lot of times, I find the obsession isn’t a desire for collaboration or anything noble, it’s as simple as this: people want to use the end products for free or very cheaply.
- Open source fine tuned assistants like LAION Open-Assistant [1]
- inference optimizations like VoltaML, FlexGenm Distributed Inference [2]
- training optimizations like Hivemind [2]
1 https://github.com/LAION-AI/Open-Assistant
2 https://github.com/underlines/awesome-marketing-datascience/...
Add the size barrier (an order of magnitude > SD for text models) and it’s pretty questionable the open source text models ecosystem is going to go anywhere.
Hardly anyone has done anything interesting with any of the existing ones and they’ve been out for longer than SD has.
Text models just aren’t that interesting to people.
Just one example, using these in games to generate unique conversations, actions, and events.
The open source models will keep up as hardware allows IMO.
Spot on. Everyone is talking about hos "fast moving" and "innovative" SD community is. Is it? I don't see many models that generate something that is NOT pretty anime girls. There are some... but if you exclude all "female focused" models, SD community doesn't seem particularly active.
People want good text models at home.
Eventually commercial entities will build pipelines which will absorb open models and build on top of them using proprietary tech, data and expensive compute.
Edit: Ah, it seems the digraph "ευ" is pronounced /ef/ in Modern Greek, despite (by convention) being transliterated "eu".
Also the ευ situation is a little more complex ie sometimes it is pronunciated as ev (depending if the next letter is a vowel).
[0]https://en.wikipedia.org/wiki/Ancient_Greek_phonology#Diphth...
> Yog-Sothoth knows the gate. Yog-Sothoth is the gate. Yog-Sothoth is the key and guardian of the gate. Past, present, future, all are one in Yog-Sothoth. He knows where the Old Ones broke through of old, and where They shall break through again. He knows where They have trod earth's fields, and where They still tread them, and why no one can behold Them as They tread.
https://twitter.com/natfriedman/status/1625850766039842824?c...
RLHF = Reinforcement Learning from Human Feedback
And yes, Eleftherios is the correct transliteration :)
So Eh-Luther AI.
See, for example: https://pzo31b.p3cdn1.secureserver.net/wp-content/uploads/20... (via https://www.fowlesbooks.com/1966-fowles-letter-to-high-schoo... )
Heres my addition! Or subtraction, i guess... (-4 chars by changing call to AdaptiveAvgPool2d):
from torch.nn import*
def c(h,d,k,p,n):S,C,A=Sequential,Conv2d,lambda x:S(x,GELU(),BatchNorm2d(h));R=type('',(S,),{'forward':lambda s,x:s0+x});return S(A(C(3,h,p,p)),*[S(R(A(C(h,h,k,1,k//2,1,h))),A(C(h,h,1)))for _ in[0]*d],AdaptiveAvgPool2d(1),Flatten(),Linear(h,n))If so, I wonder about the implications of black-box models helping to study and develop solutions for interpretability. It gives me "reflections on trusting trust" vibes.
but. didn't openai start as a nonprofit, too?