On the other hand I totally relate with the idea that it could be preferable that everyday has access to advanced AI and not just large companies and nation states.
A multimodal or segmentation algorithm is not the solution for bee-level path planning, obstacle avoidance or autonomous navigation. Getting LLMs to power a robot for household tasks with low latency to action and in an energy efficient manner is challenging enough, before talking about high-speed, highly maneuverable drones.
Not really practical at the moment of course since you can't put 8 A100s on a drone.
What is the "it" that no single entity has control over?
You have absolutely no control of what your next door neighbor is doing with open source.
Hey, if we want alcohol to be made responsibly, everyone should have their own still, made from freely redistributed blueprints. That way no single entity has control.
Great example! Yes, linux being open source has been massively beneficial to society. And this is true despite the fact that some bad guys use computers as well.
Anyone who wants to can, in fact, find blueprints for making their own still. For example, https://moonshinestillplans.com/ contains plans for a variety of different types of stills and guidance on which type to build based on how you want to use it.
And in fact I think it's good that this site exists, because it's very easy to build a still that appears to work but actually leaves you with a high-methanol end product.
Is it? I've always seen concern about methanol in moonshine but I presume it came from intentional contamination from evil bootleggers. It's difficult to get a wash containing enough methanol to meaningfully concentrate in the first place if you're making whiskey or rum. Maybe with fruit wine and hard cider there's a bit more.
The physics of distillation kind of have your back here too. The lower temperature fractions with acetone and methanol always come out first during distillation (the "heads") and every resource and distiller will tell you to learn the taste and smell, then throw them out. The taste and smell of heads are really distinctive. A slow distillation to more effectively concentrate methanol also makes it easier to separate out. But even if you don't separate the heads from the hearts, the methanol in any traditional wash is dilute enough that it'll only give you a headache.
I think it's extremely hard to build a still that appears to work but creates a high methanol end product.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2972336/#:~:tex....
It is very common for it not to be used responsibly. Alcohol causes health problems, and is implicated in accidents and violence.
Many countries have laws restricting either the sale of liquor in various ways, or its consumption in public places, or both, not to mention production and distribution.
If you don't put barriers, how quickly will AI bots take over people in online discourse, interaction and publication?
This isn't just for the sake of keeping the Internet an interesting place free of bots and fraud and all that.
But I've also heard that it's about improving AI itself. If AI starts to pollute the dataset we train AI on, the entire Internet, you get this weird feedback loop where the models could almost get worse over time, as they will start to unknowingly train on things their older versions produced.
It's entirely conceivable that even if AGI (or something comparably significant in terms of how impactful it would be to changing society or nation states) was achievable in our lifetime, it might be that:
1) Achieving it requires a critical mass of research talent in one place that perhaps currently exists at fewer than 5 companies - anecdotally only Google, Meta, and OpenAI. And a comparable number of world governments (At least in the US the best researchers in this field are at these companies, not in academia or government. China may be different.)
This makes it sound like a "security by obscurity" situation, and on a long enough timeline it may be. Without World War 2, without the Manhattan Project, and without the looming Cold War how long would it have taken for Humanity to construct a nuclear bomb? An extra 10 years? 20? 50? Hard to know. Regardless, there is a possibility that for things like AI, with extra time comes the ability to better understand and build those defenses before they're needed.
2) It might also require an amount of computing capacity that only a dozen companies/governments have.
If you open source all the work you remove the guard rails for the growth or what people focus investments on. It also means that hostile nations like Iran or North Korea who may not have the research talent but could acquire the raw compute could utilize it for unknown goals.
Not to mention that what nefarious parties on the internet would use it for. We only know about deep fake porn and generated vocal audio of family members for extortion. Things can get much much worse.
Or not, and damaging wrongheaded ideas will become a self-reinforcing (because safety! humanity is at stake!) orthodoxy, leaving us completely butt-naked before actual risks once somebody makes a sudden clandestine breakthrough.
https://bounded-regret.ghost.io/ai-pause-will-likely-backfir...
> We don’t need to speculate about what would happen to AI alignment research during a pause—we can look at the historical record. Before the launch of GPT-3 in 2020, the alignment community had nothing even remotely like a general intelligence to empirically study, and spent its time doing theoretical research, engaging in philosophical arguments on LessWrong, and occasionally performing toy experiments in reinforcement learning.
> The Machine Intelligence Research Institute (MIRI), which was at the forefront of theoretical AI safety research during this period, has since admitted that its efforts have utterly failed. Other agendas, such as “assistance games”, are still being actively pursued but have not been significantly integrated into modern deep learning systems— see Rohin Shah’s review here, as well as Alex Turner’s comments here. Finally, Nick Bostrom’s argument in Superintelligence, that value specification is the fundamental challenge to safety, seems dubious in light of LLM's ability to perform commonsense reasoning.[2]
> At best, these theory-first efforts did very little to improve our understanding of how to align powerful AI. And they may have been net negative, insofar as they propagated a variety of actively misleading ways of thinking both among alignment researchers and the broader public. Some examples include the now-debunked analogy from evolution, the false distinction between “inner” and “outer” alignment, and the idea that AIs will be rigid utility maximizing consequentialists (here, here, and here).
> During an AI pause, I expect alignment research would enter another “winter” in which progress stalls, and plausible-sounding-but-false speculations become entrenched as orthodoxy without empirical evidence to falsify them. While some good work would of course get done, it’s not clear that the field would be better off as a whole. And even if a pause would be net positive for alignment research, it would likely be net negative for humanity’s future all things considered, due to the pause’s various unintended consequences. We’ll look at that in detail in the final section of the essay.
Free software means you have to be able to build the final binary from source. Having 10 TB of text is no problem, but having a data center of GPUs is. Until the training cost comes down there is no way to make it free software.
If the training data and model training code is available then it should be considered open, even if it’s hard to train.
This will never be fully open
Making it open is the only way AI fulfills a power to the people goal. Without open source and locally trainable models AI is just more power to the big-tech industry's authorities.
I thought the big secret sauce is the sources of data that is used to train the models. Without this, the model itself is useless quite literally.
At least, that's my understanding.
Both countries have access to LLMs already. And if they didn’t, they would have built their own or gotten access through corporate espionage.
What open source does is it helps us better understand & control the tech these countries use. And it helps level up our own homegrown tech. Both of these are good advantages to have.