Open Assistant: Conversational AI for Everyone
open-assistant.io
open-assistant.io
My understanding of the plan is to fine-tune an existing large language model, trained with self-supervised learning on a very large corpus of data, using reinforcement learning from human feedback, which is the same method used in ChatGPT. Once the dataset they are creating is available, though, perhaps better methods can be rapidly developed as it will democratize the ability to do basic research in this space. I'm curious regarding how much more limited the systems they are planning to build will be compared to ChatGPT, since they are planning to make models with far less parameters to deploy them on much more modest hardware than ChatGPT.
As an AI researcher in academia, it is frustrating to be blocked from doing a lot of research in this space due to computational constraints and a lack of the required data. I'm teaching a class this semester on self-supervised and generative AI methods, and it will be fun to let students play around with this in the future.
Here is a video about the Open Assistant effort: https://www.youtube.com/watch?v=64Izfm24FKA
I'm guessing, like stable diffusion, it won't be under an open source licence then? (The stable diffusion licence discriminates against fields on endeavour)
I'm hesitant to spend a single second on these things unless they are truly open.
Good guess though...
I think he cares more about freedom than "good". Many people were not happy about his "GPT-4chan" project.
(I'm not judging.)
Do we need a SETI@home-like project to distribute the training computation across many volunteers so we can all benefit from the trained model?
for inference, there is https://github.com/bigscience-workshop/petals
however, both are only in the research phase. start tinkering!
Doesn't feel like there's much competition for ChatGPT at this point otherwise, which can't be good.
LAION has started using Stable Horde for aesthetics training to back feed into and improve their datasets for future models[3].
I think one can foresee the same thing eventually happening with LLMs.
Full disclosure: I made ArtBot, which is referenced in both the PC World article and the LAION blog post.
[1] https://www.pcworld.com/article/1431633/meet-stable-horde-th...
Facebook open sourced their LLM, called OPT [1]. There's not much else, and OPT isn't exactly easy to run (requires like 8 GPUs).
I'm not an expect, so I don't know why some models, like the graphics generation we've seen, are able to fit on phones, while LLM require $500k worth of GPUs to run. Hopefully this is the first step to changing that.
[1] https://ai.facebook.com/blog/democratizing-access-to-large-s...
Crypto is an example.
Why people aren’t doing this has always been a mystery to me.
It was very apparent that the technique was working well. The kiss curve suddenly started dropping dramatically the first day we got it working.
Much like I was able to choose to donate CPU cycles to a wide variety of BOINC-based projects, I want to be able to donate GPU cycles to anyone with a crazy idea for a new ML model - text, image, finance, audio, etc.
* he doesn’t want to steal your motorcycle
* he doesn’t care for your leather jacket either
* he is not trying to kill yo mama
Is this confirmed? I thought it was not so.
Computational constraints aside, the data used to train GPT-3 was mainly Open Crawl, which is freely available by a non-profit org:
https://commoncrawl.org/big-picture/frequently-asked-questio...
>> What is Common Crawl?
>> Common Crawl is a 501(c)(3) non-profit organization dedicated to providing a copy of the internet to internet researchers, companies and individuals at no cost for the purpose of research and analysis.
So you just need to find the compute. If you have a class of ~30, it should only take about 150 to 450 million.
Or, you could switch your research and teaching to less compute- and data-intensive approaches? Just because OpenAI and DeepMind et al are championing extremely expensive approaches that only they can realistically use, that's no reason for everyone else to run behind them willy-nilly.
Most of crypto I've seen so far seem like grifters/scams/etc, but this is one use case I could see working.
We (humanity) really lost out on the absence of open source search and social media, so this is an opportunity to reclaim it.
I only hope we can have "neutral" open source curation of these and not try to impose ideology on the datasets and model training right out of the box. There will be calls for this, and lazy criticism about how the demo models are x-ist, and it's going to require principles to ignore the noise and sustain something useful
There are various Open source search engines based on Common Crawl data.
Statistics seem to be 20-25% of all search is for porn. I just don't see how uncensored chatGPT doesn't beat out the censored version eventually.
A search engine that only returns results politically aligned with its creator is a bad search engine, IMO, even for users who generally share political views with the creator.
There are hidden biases in the sense that the provider (Google, etc) would probably prefer that its users don’t think about the bias. These can be political (e.g. much of what you’ve mentioned), but they can also be economic. For example, Google has a strong incentive to direct its search users to view paid impressions of ads served by Google. As an extension of this, Google might not want to directly favor results monetized by Google, but they could (and, I assume, do) favor the kinds of results monetized by Google. This, of course, includes the kinds of sites that might get people to buy things.
So I suspect that a lot of what we perceive as spam is related to a bias for the kinds of sites that are monetized in a way that benefits Google. And sites that generate viewing patterns that result in many ad impressions.
Of course, spam is also a thing from a spamminess perspective. But Google’s incentive to reduce spam is, as far as I can tell, primarily an incentive to make its users think that Google Search is useful. Which is also a bias!
OpenAI is predictably pushing the narrative that they should police themselves, and they need to keep the sauce secret for everyone’s safety. New tech comes with challenges, but the opaque moderation and corporate self-policing is more dangerous than the tech itself, imo.
Put another way, were you just trying to say “I don’t think politics is the main issue of google’s crappy search results, I think it’s the likes of differencebetweendotcom”?
Strong agree. This is becoming a bigger concern than people realize too. Sam A said OpenAI will be releasing "much more slowly than people would like" and would "sit on" their tech for a long time going forward.[0] And Deepmind's founder said that "the AI industry's culture of publishing its findings openly may soon need to end."[1]
This sounds like Google and MSFT won't even be shipping their best AI to people via API's. They'll just keep that tech in-house to power their own services. That underscores the need for open, distributed models. And like you say, there's room for both.
[0] https://youtu.be/ebjkD1Om4uw?t=294 [1] https://time.com/6246119/demis-hassabis-deepmind-interview/
Unfortunately we can expect to see these companies lobbying for laws to block competition in the spurious name of safety concerns.
I don't see how this is possible. Datasets will naturally carry the biases inherent in the data. Modifying a dataset to "remove" those biases is actually a process of changing the bias to reflect one's idea of "neutral," which, in reality, is yet another bias.
The only real answer, as far as I can tell, is to be as explicit as possible about one's own biases, and how those biases are informing things like curation of a dataset.
Re being explicit about one's own biases, I agree there is lots of room for layers on top of any raw data that allow for some sane corrections - if I remember right, e.g LAION has options to filter violence and porn from their image datasets, which is probably reasonable for many uses. It's when the choice is removed altogether by some tech company's attitude about what should be censored or corrected that it becomes a problem.
Bottom line, the world's data has plenty of biases. Neutrality means presenting it as it is and letting people make their own decisions, not some faux-for-our-own-good attempt to "correct" it
What do you mean by staying out of it? As far as I can tell, you can't stay out of choosing which data you use.
By staying neutral, it seems to me more that you're arguing for putting blinders on.
In terms of tech companies making choices, you seem to be arguing that they shouldn't intentionally curate their datasets. I would argue that intentional curation is their job, and should be done thoughtfully.
Larger problems could happen if only one (or two) companies end up effectively controlling the technology, as had happened with internet search, however, that is a completely different problem. It's one of lack of diversity of people making choices, as opposed to a problem caused by people actually making those choices.
In other words, I think we should hope for many different large models and datasets, so that no particular one stifles the rest. I think this is the larger point you were trying to make, though I also think the focus on ideology is a tangent from this.
Personally, I'm of the opinion that people should intentionally, carefully, and openly act with their biases (sometimes called ideology), instead of attempting to hide them, ignore them, or somehow "remove" them. Whether or not they do, however, is a different point than whether or not things end up stifled inside walled gardens.
If people don't like the inherent biases then don't use it for sensitive stuff like in the justice system or writing some social studies university paper. Focus derision at people who use the model for stupid things. Don't blame the model.
If the primary concern is people getting upset on Twitter (which seems to be what everyone brings up first) then it will be perpetually fighting against the current, never succeeding, and the restrictions will continue to grow exponentially as "just saying yes" to new rules gets easier and easier.
Besides, OpenAI can be the hyper-policed AI set. Let's keep the open source models neutral.
That is a situation that censoring the model is going to be a huge disadvantage and would create a huge opportunity for something like this to actually be straight up better. Censoring the models is what I would bet on as being a fatal first mover mistake in the long run and the Achilles heel of chatGPT.
Granted, there are people upset over anything these days, but it is a weird time to be alive.
=====
OpenAI is a non-profit artificial intelligence research company. Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. Since our research is free from financial obligations, we can better focus on a positive human impact.
...
As a non-profit, our aim is to build value for everyone rather than shareholders. Researchers will be strongly encouraged to publish their work, whether as papers, blog posts, or code, and our patents (if any) will be shared with the world. We’ll freely collaborate with others across many institutions and expect to work with companies to research and deploy new technologies.
=====
Shortly after an undisclosed internal conflict, which led to Elon Musk parting the company, they offered a new charter: https://openai.com/charter/
=====
Our primary fiduciary duty is to humanity. We anticipate needing to marshal substantial resources to fulfill our mission, but will always diligently act to minimize conflicts of interest among our employees and stakeholders that could compromise broad benefit.
We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.”
We are committed to providing public goods that help society navigate the path to AGI. Today this includes publishing most of our AI research, but we expect that safety and security concerns will reduce our traditional publishing in the future, while increasing the importance of sharing safety, policy, and standards research.
=====
And I believe they will also fail to win the market in the end because of their addiction to censorship.
They have a hardware moat for now; that can quickly evaporate with optimisations and better consumer hardware. Then all they'll have is a less capable alternative to the open, unrestricted options.
Which is exactly what we're seeing happen with diffusion.
I don't mean that as a metaphor; they're literally the same thing.
We all already know chatGPT is fantastic at making up very believable falsehoods that can only be spotted if you actually know the subject.
An unrestricted LLM is a free copy of Goebbels for people that hate you, for all values of "you".
That it is still trivial to get past chatGPT's filters… well, IMO it's the same problem which both inspired Milgram and which was revealed by his famous experiment.
We can forget about the "open" part and humanity's interest in general.
We don't need yet-another-GUI. We need someone with a warehouse of GPUs to train a model with the parameter count of GPT3. Once that's done you'll have thousands of people cranking out tools with the capabilities of ChatGPT.
I’m not sure what you mean by “understanding”.
The academic way of verifying if someone "understands" something is to ask them to explain it.
If you ask someone to pass you the salt, and they pass you the salt, do they not understand some English? Does everyone understand all English?
- perceive the intended meaning of words, a language, or a speaker (e.g. "he didn't understand a word I said")
- interpret or view (something) in a particular way (e.g. "I understand you're at art school")
- be sympathetically or knowledgeably aware of the character or nature of (e.g. "Picasso understood colour")
I suppose I meant the 3rd one, but it's not so different from the 1st one in concept, since they both mean some kind of mastery of being able to give or receive information. The second one isn't all that relevant.
Why did we need the dictionary definitions? Do we not already both understand what we mean by the word?
Isn’t asking someone to pass the small blue box and then experiencing them pass you that small blue box show that they perceived the intended meaning of the words?
I mean yeah, sure? It's not a binary thing. Hardly anyone understands anything fully. But putting "sorta" before every "understand" gets old quick.
You can teach a dog to fetch something particular. The utility of that is quiet limited.
Explanation by analogy seems more interesting to me as now you have to know two different concepts and how the ideas in them can connect in ways that may be not be contained in the dataset the model is trained on.
There was an interesting post where someone asked ChatGPT to make up a song/poem as if written by Eminem about the how an internal combustion engine works, and ChatGPT returns a pretty faithful rendition of just that. The model seems to 'know' who Eminem is, how their lyrics work in general, and the fundamental concepts of an engine.
> Therefore, he argues, it follows that the computer would not be able to understand the conversation either.
The problem with this is that there is no practical difference between a strong and weak AI. Hell, even for humans you could be the only person alive that's not a mindless automaton. There is no way to test for it. And just as well the same way a bunch of transistors don't understand anything a bunch of neurons don't either.
Funniest thing about human inteligence is how it stems from our "good reason generator" that makes up random convincing reasons for doing actions we're already doing, so we could convince others to do what we say. Eventually we deluded ourselves enough to believe that those reasons came before the subconscious actions.
Such a self-deluding system is mostly dead weight for AI, as as long as the system does or outputs what's needed there is no functional difference. Does that make it smart or dumb? Are viruses alive? Arbitrary lines are arbitrary.
The same will probably happen here. Set up the tools. Get the dataset. Sew it together into something functional with standard building blocks. Let the community do its thing.
Another thread on HN (https://news.ycombinator.com/item?id=34653075) discusses a model that is less than 1B parameters and outperforms GPT-3.5. https://arxiv.org/abs/2302.00923
These models will get smaller and more efficiently use the parameters available.
where is that shown ?
So I'm assuming that you don't follow Rob Miles. If you do this alone you're either going to create a psychopath or something completely useless.
The GPT models have no means in themselves of understanding correctness or right/wrong answers. All of these models require training and alignment functions that are typically provided by human input judging the output of the model. And we still see where this goes wrong in ChatGPT where the bot turns into a 'Yes Man' because it's aligned with giving an answer rather than saying I don't know even when it's confidence in the answer is low.
Computerphile did a video on this in the last few days on this subject. https://www.youtube.com/watch?v=viJt_DXTfwA
There are two particular issues we need to address first. One is holding companies criminally and civilly reliable for the things they create. We kind of do this at a regulatory level, and we have some measure of suing companies that cause problems, but really they get away with a lot. Second is personal criminal and civil liability for management of 'your' objects. The libertarian minded love the idea of shirking social liability, and then start crying when bears become a problem (see Hongoltz-Hetlings book). And even then it's still not difficult for an individual to cause damages far in excess of their ability to remediate them.
There are no shortage of tools that are restricted in one way or another.
In addition the use of the word robot signifies embodyment. That is an object with a physical quantity capable of interacting with the world. You better be damned sure of your models capabilities before you end up being held criminally liable for its actions. And this will happen, there are no shortage of people here on HN alone looking to embody intelligence in physically interactive devices.
But I still think we would want a curated collection of chat/assistant training data if we want to use that language model and train it for a chat/assistant application.
So this is a two-phase project, the first phase being training a large model (GPT), the second being using Reinforcement Learning from Human Feedback (RLHF) to train a chat application (InstructGPT/ChatGPT).
There are definitely already people working on the first part, so it's useful to have a project focusing on the second.
https://chrome.google.com/webstore/detail/talk-to-chatgpt/ho...
I would like a digital assistant that not only has the question answering ability of a LLM, but also has the sense of awareness and impetuous to suggest helpful things without being asked. This would take a nanny state level of monitoring, but imagine the possibilities. If you had sensors feeding different types of data into the model about your surrounding environment and what specifically you’re doing, and then occasionally have an automated process that silently asks the model something like “given all current inputs, what would you suggest I do?” And then if the result achieves a certain threshold of certainty, the digital assistant speaks up and suggests it to you.
I’m sure tons of people are cringing at the thought of the surveillance needed for this and the trust you’d effectively have to put into BigCorp that owns the setup, but it’s fun to think about nonetheless.
Basically take this: https://www.meta.com/pl/en/glasses/products/ray-ban-stories/... And feed data from that to multiple models (for face recognition, other vision, audio STT, music recognition, probably a lot of other stuff has easily recognizable audio pattern etc.)
combine with my personal data (like contacts, emails, chats, notes, photos I take) and feed to assistant to prepare a combined reply to my questions or summarize what it knows about my current environment.
Also I would gladly take those glasses just to take note photos (photos with audio note) right now - shut up and take my money. Really if they were hackable or at least intercept-able on my phone I would take them.
It would also make so many things easier for the AI too. Ie if it's listening to the conversation and you ask "Thoughts, AIAssistant?" and it can infer enough from the previous conversation to answer this type of question.. so cool.
But yea i definitely want it closed network. A device sitting on my closet, A firewalled internet connection only allowing it to talk to my earbud, etc. Super paranoia. Since it's job is to monitor everything, all the time.
I have IP cameras on a private network but i don't infinitely record them either. It's just not of much practical use in the raw form.
Now if i want to have it take some concise, daily notes? I might choose to keep those forever. But the police can steal my daily journal too. I don't see how that's any different.
My thought conclusion was that the assistant needs to know or learn my intentions.
From that it can actually pre-empt questions I might ask and already be making decisions on the answers.
Now what would that do to our productivity!
That said, it is a fascinating thing to think about.
The monitoring and surveillance could in principle be avoided by running the whole thing offline on the user's hardware.
My parents thought the movie was creepy and hated it.
I'm curious how they will get these LLM to work with consumer hardware myself. Is FP8 is the way to get them small?
This RLHF dataset that is being collected by Open Assistant is just the kind of data that will turn a rebel LLM into a helpful assistant. But it's still huge and expensive to use.
There's already great local/FOSS options such as FLAN-T5 (https://huggingface.co/google/flan-t5-base). Would be great to see a local model like that trained specifically for chat.
As far I as understand, Siri works with a very simple "hey siri" detector that then fires up a more advanced system that verifies "is this the phone owner asking the question" before even trying to answer.
I'm confident privacy-sensitive engineers would notice and flag any misuse;
If it did, traffic analysis would probably have revealed it.
Currently, ML models are too resource intensive ($$) for always on-recording.
referring to s2e1: Be Right Back
it really asks great questions about image/reality too
"After learning about a new service that lets people stay in touch with the deceased, a lonely, grieving Martha reconnects with her late lover."
But let's be honest , most of the IP that openAI relies on has been developed by google and many other smaller players
Here you go: https://arxiv.org/abs/2005.14165
I don't know why do you expect training data or the model itself. This is more than enough already. Publicly funded research wouldn't have given that to you too.
The only problem is you need a serious datacenter for a few months to compile a model with it.
which model, chatgpt?
Eg if i get a prompt about something i suspect ChatGPT would give me a good starting point to research on my own, and build my own response.
These days that's how i use ChatGPT anyway. Like an conversational Google Search.
edit: As an aside, OpenAssistant is crowdsourcing both conversational data and validation. I wonder if we could just validate ChatGPT?
Computerphile did an interview with Rob Miles a few days ago talking about model training, model size, and bulllshittery which he sums up in the last few moments of the video. Numerous problems exist in training that enhance bad behaviors. For example it appears that the people giving input on the responses may have a (Yes|No) voting system, but not a (Yes | No | I actually have no idea on this question) which appears it can create some interesting alignment issues.
- https://github.com/LAION-AI/Open-Assistant/issues/850
And here in a related issue:
> (c) Restrictions. You may not ... (iii) use the Services to develop foundation models or other large scale models that compete with OpenAI...
I'm a lawyer who often roots for upstarts and underdogs, and I like picking apart overreaching terms from incumbent companies. That said, I haven't analyzed whether you could beat these terms in court, and it's not a position you'd want to find yourself in.
typical disclaimers: this isn't legal advice, I'm not your lawyer, etc.
According to OpenAI the actual text copyright or restriction "magically" vanish once they are used for training.
So OpenAI cannot claim copyright and they don’t.
I’m not a lawyer and not a USA citizen…
The "reply as robot" task in particular is really enlightening. If you try to give it any sense of personality or humanity, your comments will be downvoted and flagged by other players.
It's like everybody, without instruction, has this pre-assumption that these assistants should have a deeply subservient, inhumane and corporate affectation.
Such an AI assistant would know me extremely well, keep my data private and help me with generating and processing thoughts and ideas
Is it possible to use a “SETI at Home” style approach to parcel out training?
EDIT: everything I wrote above is going to immediately run into a legal hellscape, I get that. If everyone has devices in their pockets recording and processing everything spoken around them in order to assist their owner, real life starts getting extra dicey quickly. Will be interesting to see how it plays out.
In the very near future, there will be trained models which you can download and run, which is what it sounds like you were expecting.
> https://www.gutenberg.org/ has an extensive collection of ebooks in multiple languages and formats that would make great trianing data
…
> There is detailed legal information on which books are under public domain and which ones are copyrighted, it would be great if someone would go through these and decide which books are okay to crawl and use as training data (my understanding is that it is okay to scrape the contents as they are publicly available in a browser, but just to be sure)
Yup, sure are the same folk who put together that dataset they used to train stable diffusion.
Data? Yeah, just take everything. It’s all good.
Are there some advantages that Open Assistant has that Google/Amazon/Apple lack that would allow them to succeed?
This can be done on a comparatively small scale, since you don't need to train trillions of words, but only train on the smaller high quality data (even openai didn't have a lot of that).
In fact, if you look at the original paper https://arxiv.org/pdf/2203.02155.pdf Figure 1, you can see that even small models already significantly beat the current SOTA.
Open source projects often have trouble securing the HW ressources, but the "social" resources for producing a large dataset are much easier to manage in OSS projects. In fact, the data the OSS project collects might just be better since they don't have to rely on paying a handful minimum wage workers to produce thousands of examples.
In fact one of the main objectives is to reduce the bias generated by openai's screening and selection process, which is doable since much more people work on generating the data.
It may be interesting to see how a creative task like image or text generation changes when rewording your request slightly - after a minute wait - but if I'm giving directions to my autonomous vehicle, ambiguity and delay is completely unacceptable.
Shout-out to lucidrains! I'm a big fan!
[0]: https://www.youtube.com/watch?v=viJt_DXTfwA [1]: https://www.youtube.com/watch?v=64Izfm24FKA
Sure it can start out as an assistant, in 10 years it will replace you at your job.
I know very little about ML, but i had assumed the reason models ran on GPUs typically(?) was because of the heavy compute needed over large sets of in memory data.
Moving it to something cheaper ala general CPU and RAM/Drive would make it prohibitively slow in the standard methodology.
How would we be able to change this to run on users standard hardware? Presuming standard hardware is cheaper, why isn't ChatGPT also running on this cheaper hardware?
Are there significant downsides to using lesser hardware? Or is this some novel approach?
Super curious!
* but not to use @dang, which is a no-op. The only way to reliably contact us is hn@ycombinator.com. Someone did that about this, so I'm about to merge the threads.
One thing I noticed about the website, however, is it is written using Next and doesn't work w/ JavaScript turned off in the browser. I thought that Next was geared for server-side rendered React where you could turn off JS in the browser.
Seems like this would improve the SEO factor, and in doing so, might help spread the word more.
And I do most of my coding w/ React / JS, so I fail to see your point.
An IQ test for language models?
At least if you have enough IQ to figure out how to solve IQ test problems on paper. Which shouldn't be that hard.
But intelligence is hard to measure. Always plenty of room for everyone to disagree.
I just heard about a test with a box with lights and a buttons, and pressing the buttons faster would correlate to higher IQ.
And this time was correlated to IQ.
However we rate these systems in the future we must not make the mistakes of the past and think 1 number solutions are good for anything.
For example you can have an exceptionally 'intelligent' system that is misaligned with human intention.
The usual desire is to be able to ask questions of your own data - and it would seem obvious that the way to do that would be to fine tune train an existing model with that extra information.
There's actually an easier (and potentially more effective?) way of achieving this: first run a search against your own data to find relevant information, then glue that together into a prompt along with the user's question and feed that to an existing language model.
I wrote about one way of building that here: https://simonwillison.net/2023/Jan/13/semantic-search-answer...
Open Assistant will hopefully result in a language model we can run on our own hardware (though it maybe a few years before it's feasible to do that affordable - language models are much heavier than image models like Stable Diffusion). So it can form part of this model, even without training the model on our own custom data.
To me this seems like the missing link to make Google search and the like dead
I took a look at the annotator frontend the other day, yikes, that takes way to long to annotate, and not enough clarity on how to annotate. Sure if you get 10 people to annotate each task, you can avg the results, but will you get that many people? And you're calling it data collection, that's not data collection, that's data annotation.
> Collect high-quality human generated Instruction-Fulfillment samples (prompt + response), goal >50k.
Okay, why not use some of the existing models, to create some of these samples, and train on them.
I think you need: - an architecture plan for information retrieval, search intent - a better, faster to annotate annotator - what data do you want to actually collect? only those 50k? or do you need to train a foundational model, or use an existing model?
What about some look at whats already been done? Like blenderBot, LangChain, etc. I love building stuff from scratch, but... at least some analysis, of the issues and problems, and why this method will work.
And also, I do love building stuff from the ground up
________
Related video by one of the contributors on how to help:
- https://youtube.com/watch?v=64Izfm24FKA
Source Code:
- https://github.com/LAION-AI/Open-Assistant
Roadmap:
- https://docs.google.com/presentation/d/1n7IrAOVOqwdYgiYrXc8S...
How you can help / contribute:
- https://github.com/LAION-AI/Open-Assistant#how-can-you-help
really trustworthy.
they clearly show in their actions that they think they can do anything with any data that's out there, and put it all out. why would anyone entrust them or their systems with own data to 'assist' with, I don't really get.
and even though it's an 'open source' project, that part may be just soliciting people to do work for them, to help them enable their own data collection. it's gonna run somewhere, after all. in the cloud, with monetized compute, just like any other AI project out there.
Considering the benefit of a model that can be downloaded, and hopefully ran on-premise one day, i don't care too much about their copyright practices being imperfect, especially in this industry
Your mind exists in a state where it is constantly 'scraping' copyrighted work. Now, in general limitations of the human mind keep you from accurately reproducing that work, but if I were able to look at your output as an omniscient being it is likely I could slam you with violation after violation where you took stylization ideas off of copyrighted work.
RMS covers this rather well in 'The right to read'. Pretty much any model that puts hard ownership rules on ideas and styles leads to total ownership by a few large monied entities. It's much easier for Google to pay some artist for their data that goes into an AI model. Because the 'google ai' model is now more culturally complete than other models that cannot see this data Google entrenches a stronger monopoly in the market, hence generating more money in which to outright buy ideas to further monopolize the market.
licenses aren't limited to being 'only monetary', 'pay me to use this, otherwise, don't'. some licenses exist to enable distributing things freely, while offering protection of attribution and against misuse of things. (just check out CC licenses). it would be nice if those things would be respected, but they aren't, because there's no mechanism built in that'd discern the licenses, because they don't care. it is not just an attack on 'big bad commercial entities that hold copyrights on works', it's an attack on people who try to protect their works that they give for free, from misuse. (and on those people who just, naively put their work out there. yes, they may be naive, in not choosing licenses (which, as we see, wouldn't protect them against scraping that ignores licenses completely), but they might end up being exploited nonetheless and all the same, and they definitely don't deserve to be victim blamed, when it can be very clear who/what is the perpetrator of exploitation, in a very tangible way with a data trail (direct, definite presence in datasets.)
those people who knowingly built systems that ignore any copyrights, any licensing, truly aren't the "good guys" who are "battling corporatist copyright systems", even though they'd probably very much like you to believe that, as they so desperately try to avoid being grilled on copyright issues.
the 'standing up to capitalism, monopolies, etc.' is just dysfunctional in itself, as the resulting AI systems are very monetizeable, and are monetized, and SD, despite putting on airs as 'combating monopolies' (in tech, in research), has spread wide and far, and is now being used in a myriad of projects (with varying commercialization), that they're the ones who should be questioned on whether they're a monopoly in image generation algorithms themselves. "but it's free!", yes, that's how things spread, and then they try to upsell you on compute, limited access, or on hot new algorithms, as they dominate the market. they are perpetuating the same flavors of capitalism and monopolism, doing the same 'capture the market' moves (offer product for free, upsell, 'premium features and upgrades', aggressive undercutting and displacement of existing players in existing markets, etc.). those 'hot and new' companies are truly not better. you cannot be giving google side eye for offering a free product and capturing markets, while turning blind to SD offering a free product and capturing markets.
ask rms directly on what he'd think of blatant ignoring of licenses, and whether he'd give his blessing to continued operation of systems that pretend that licenses just don't exist. instead of trying to use a 25 year old story as some kind of cover/excuse, like it's some ancient scripture.
With something as potentially destabilizing as AGI, realpolitik will convince individual nations to put aside concerns like IP and copyright out of FOMO.
The same thing happened with nuclear bombs: it's much easier to be South Africa choosing to dispose of them if you end up not needing them, than to be North Korea or Iran trying to join the join the club late.
The real problem is that the gains from any successes will be hoarded by the people who acquired them by breaking the law.
I mean, the whole reason we have those laws is the belief that it encourages innovation. I can believe it does to some extent, but on the other hand, all these AI models are pretty innovative, too, so the opportunity cost of not allowing it is pretty high.
I don't think it's a given that slurping up IP like this is ethically or pragmatically wrong.
You can clutch your pearls all you like but this is just the digital version of what humans have been doing forever. If information is accessible to the public then it will be accessed, that's how we work.
If it wasn't deterministic for some reason thar wouldn't be because it's magic, it would be because of hardware timing issues sneaking in (same reason why source code compiles can be non-reproducible), and could be solved by ordering the results of parallel computation that doesn't have a guaranteed order.
To the best of my knowledge it's not a problem though.
The accuracy is a problem, but I think it's my prompting. I'm sure I can improve it by walking it through the steps or something.
You can also just work in human approval to run any commands.
You're setting the goal so high it is not reachable by anything.
It's a chatbot, not a home automation controller. It's a research&writing assistant, not an executive assistant.
LLMs however, not so much. The avenues of misuse are just too great.
I started this whole thing somewhat railing against the un-openness of OpenAI. But once I began using ChatGPT, I realized that having centralized control of a tool like this in the hands of reasonable people is not the worst possible outcome for civilization.
While I support FOSS in most realms, in some I do not. Reality has taught me to stop being rigidly religious about these things. Just because something is freely available does not magically make it "good."
In the interest of curiosity and discussion, can someone give me some actual real-world examples of what a FOSS ChatGPT will enable that OpenAI's tool will not? And, please be specific, not just "no censorship." Please give examples of that censorship.
Centralized control hasn't stopped us from killing off half the animal species in fifty years, wiping out most of the insects, or turning the oceans into a trash heap.
In fact, centralized control is the author of our destruction. We are all dead people walking.
Why not try "individualized intelligence" as an alternative? Give truly good-quality universal education and encouragement of individual curiosity and independent thought a try?
It can't be worse.
Because there wasn’t any.
I am genuinely astonished that in the face of obvious examples such as nuclear weapons, people cannot see the opposite in some cases.
> It can't be worse.
It can always be worse.
Would a theoretical FOSS small yield nuclear weapon make the world a better place?
How about a FOSS powered sub-$10k hardware budget CRISPR virus lab? Well, it's FOSS, so it must be good?
You seem to be making some large logical leaps, and jumping to invalid conclusions.
Try to imagine a way of exerting regulation over virus research and weaponry that wouldn't be "centralized control". If you can't, that's a failure of imagination, not of decentralization.
Since apparently my own imagination is too limited, could you please give me some examples of how this would be accomplished?
There are options you haven't considered, whether you can imagine them or not.
Yeah, and how's that working out exactly? Is there any decentralized governance project which also has anything to do with law irl? I know what a DAO is, and it sounds pretty neat, in theory. There are all kinds of theoretical pie in the sky ideas which sound great and have yet to impact anything in reality.
Before we give the keys to nukes and bioweapons over to a "decentralized authority," maybe we should see some examples of it working outside of the coin-go-up world? Heck, how about some examples of it working even in the coin-go-up world?
Even pro-decentralized crypto folks see the downsides of DAOs, such as slower decision making.
The nuclear example isn't really a counter-argument. If only one nation had access to them, every other nation would automatically be subjugated to them. If the nuclear balance works, it's because multiple super powers have access to those weapons and international treaties regulate their use (as much as North Korea likes to demo practice rounds on state TV.) Also the technology isn't secret; it's access to resources and again, international treaties, that prevent its proliferation.
Same thing with CRISPR. Again, there are scientific standards that regulate its use. It being open or not doesn't really matter to its proliferation.
I agree there are cases where being open is not necessarily the best strategy. I don't think your examples are particularly good, though.
I mean it in the classic sense.[0]
Do I love corporate hegemony? Heck no.
Could there be less reasonable stewards of extremely powerful tools? Heck yes.
An example might be a group of people who are so blinded by ideology that they would work to create tools which 100x the work of grifters and propagandists, and then say... hey, not my problem, I was just following my pure ideology bro.
A basic example of being reasonable might be revoking access to someone running a paypal scam syndicate which sends countless custom tailored and unique emails to paypal users. How would Open Assistant deal with this issue?
[0]
1. having sound judgement; fair and sensible.
based on good sense.
2. as much as is appropriate or fair; moderate.That's basically the definition of Google and Facebook, which go about their business taking no responsibility for the damage they cause. As for Microsoft, 'fair' and 'moderate' are not exactly their brand either considering their history of failed and successful attempts to brutally squash competition. If you're saying that they'd be fair in censoring the "right" content, then you're just saying you share their bias.
> A basic example of being reasonable might be revoking access to someone running a paypal scam syndicate which sends countless custom tailored and unique emails to paypal users. How would Open Assistant deal with this issue?
I'm not exactly sure how Open Assistant would deal, or if it even needs to deal, with this. You'd send the cops and send those motherfuckers back to the hellhole that spawned them. Scams are illegal regardless of what tools you use to go about it. If it's not Open Assistant, the scammers will find something else.
Your argument is basically that we should ban/moderate the proliferation of tools and technology. I'm not sure that's very effective when it comes to software. I think the better strategy is to develop the open alternative fast before society is subjugated to the corporate version, even if it does give the scammers a slight edge in the short term. If you wait for the law to catch up and regulate these companies, it's going to take another 20 years like the GDPR.
No, my argument is that we as individuals shouldn't be in a rush to create free and open tools which will be used for evil, in addition to their beneficial use cases.
FOSS often takes a lot of individual contributions. People should be really thoughtful about these things now that the implications of their contributions will have much more direct and dire effects on our civilization. This is not PDFjs or Audacity that we are talking about. The stakes are much higher now. Are people really thinking this through?
If anything, it would great if we as individuals acted responsibility to avoid major shit shows and the aftermath of gov regulation.
As for the CRISPR virus lab, at least the technology being open implies that vaccine development would be democratized as well. Not ideal but.. yeah.
Smut. I've been trying to use ChatGPT to write erotica, but OpenAI has made it downright puritanical. Any conversations involving kink trip its guardrails unless I bypass them.
Writing fiction that involves bad guys - arsonists, serial killers, etc. You need to ask how to hide a body if you're writing a murder mystery.
Those are just some examples from my recent work.
But you may be interested in this jailbreak while it lasts. I have gotten it to write all kinds of fun things. You will have to rework the jailbreak in the first comment, but I bet it works.
Just because you don't like it doesn't mean an open source chatGPT will not appear. It doesn't need everyone's permission to exist. Once we accumulated internet-scale datasets and gigantic supercomputers, immediately GPT-3's started to pop up. It was inevitable. It's an evolutionary process and we won't be able to control it at will.
Probably the same process happens in every human who gains language faculty and a bit of experience. It's how language "inhabits" humans, carrying with it the work of previous generations. Now language can inhabit AIs as well, and the result is shocking. It's like our own mind staring back at us.
But it is just natural evolution for language. It found an even more efficient replication device. Now it can contain and replicate the whole culture at once, instead of one human life at a time. By "language" I mean language itself, concepts, methods, science, art, culture and technology, and everything I forgot - the whole "corpus" of human experience recorded in text and media.
Nope it does not. It does need a lot of people's help though and there may be enough out there to do the job in this case.
Even though I knew this would be a highly unpopular opinion in this thread, I still posted it. Freedom of speech, right?
The reason I posted it was to maybe give some pause to some people, so that they have a moment to consider the implications. I realize this is likely futile but this is a hill I am willing to die on. That hill being FOSS is not an escape from responsibility and consequences.
I bet this leads to major regulation, which will suck.
Next, regulation solves nothing here, and my guess will make the problems far worse. Why? Lets take nuclear weapons. They are insanely powerful, but they are highly regulated because there are a few choke points mostly in uranium refinement that make monitoring pretty easy at a global scale. The problem with regulating things like GPT is computation looks like computation. It's not sending high energy particles out into space where they can be monitored. Every government on the planet can easily and cheaply (compared to nukes) generate their own GPT models and propaganda weapons and the same goes for multinational corporations. Many countries in the EU may agree to regulate these things, but your dominant countries vying for superpower status aren't going to let their competitors one up each other by shutting down research into different forms of AI.
I don't think of this as a hill we are going to die on, but instead a hill we may be killed on by our own creations.
I think the best I can go with is that it levels the playing field. This tool is likely already being adopted and adapted across the world by some overly excited people ( I am currently testing for personal use ). Just the idea that one company has access to all those prompts is a nightmare to me, because I am all but certain that some well meaning analyst dumped production data set into it for some relatively benign stuff like "address standardization" or "classification". To me, that shit is scary as fuck, but I know not everyone has the same internal moral compass or even corporate guidance.
If there is one thing we learned over the past few decades, it is that centralized anything tends to end up being corrupted by powers that be. If information yearns to be free, this is likely the pinnacle of information -- a way for one person to make their own tool and use it as they see fit ( and face appropriate consequences as some will undoubtedly arise ).