Open source AI is critical – Hugging Face CEO before US Congress
venturebeat.com
venturebeat.com
Sam Altman, on the other hand? He's the figurehead of a consumer-facing product with a brilliant marketing team and probably a crack team of lobbyists to boot. He's got billions of dollars in Microsoft funding and a product you can go use for free. All he's got to do is roll in there first with an AI alarmist response and convince lawmakers the state-of-the-art is dangerous. After all, OpenAI is synonymous with SOTA, right? So we follow the industry leaders, without really looking where they're headed. And now we end up here.
Oh how I wish Open Source software was fairly represented in court. But ideas don't pay for themselves, and money talks awful loud...
Everything after this point will just be habitually disappointing. Everything will be held to the impossibly high standard of the first time you tried ChatGPT, not the more realistic benchmarks of GPT-2, LLaMA, Bert, or even Talk-To-Transformer. I don't think it will be possible to recapture that magic, and Sam Altman was the first-mover on turning that awe into fear.
Instead of an oracle they released an improv comedian. Entertaining enough but not what you want running the show.
Anything local bypasses traversal of any network edges they can tap and monitor. Anything you ask Google is logged, and your search history can be subpoenaed, same as with any public library and the books you check out. OpenAI or whatever gatekeeper would assume that role and charge tolls.
Do the same research with Vicuna-13B-ggml or whatever and nobody can sniff out what you're doing. Which sounds great for people who just want to jerk off and be left alone, but has terrifying implications for anybody tasked with stopping more-malicious users.
Unfortunately nobody can ever compromise on these things. Best case would be adopting a law like the recent one in NL, where given probable cause the government can hack into your machine, but that won't happen. The only alternative in their favor is banning local models and open-source development of them altogether.
People yearn for "a room of their own"[1], a sanctuary for contemplation and creativity. There was a time when our minds and pieces of paper were the only canvases for our imaginations, but increasingly our web browsers have morphed into thought surveillance systems. Now, the tide is turning back towards private creative spaces, enhanced by generative AI. Even if there are attempts to ban it, the edge now lies with private AI. In the future, we might even have the ability to run even more powerful models on standard hardware.
Moreover, we must consider the defensive strategies against other AI entities that will soon permeate the internet. Operating much like a firewall, private AI models could safeguard users from external threats such as involuntary data breaches and misinformation. This will be an endless cat-and-mouse game akin to battling security exploits or viruses—there's no one-size-fits-all solution. Venturing into the internet without your AI shield may soon be as unthinkable as stepping outside without a mask amidst a pandemic like COVID.
edit: you're talking about crypto in general. Bitcoin spam has broken my brain.
Nothing open approaches GPT-4 yet, which I have heard is an overlay of multiple 220B models.
The only way you will see that on even high end consumer hardware is advances in compression and pruning, but those are happening. There’s a lot of research in those areas.
Of course then cloud models could use the same techniques to get even bigger, if the model and training will keep scaling.
Edit: as AI matures you may also see lots of custom ASICs hit the market. I could see those getting cheap enough to run many hundred B models locally. The acceleration of an ASIC designed for a specific type of model over CPU and GPU could be dramatic, reducing the amount of silicon you need.
Mr Altman didn't go before congress to convince them of anything, he was invited by Blumenthal to make their case to the media. Congress is embarrassed that social media got out of hand and is chomping at the bit to regulate AI. There was no disagreement at the testimony, the intentions of government and industry are aligned perfectly here - roll out the red carpet for centralized control, regulate competition out of existence.
AI is an important but ultimately small part of this overall transformation. Let me remind everybody that bitcoin, aiming to disrupt the very fundamentals of economic life, is also open source.
The evolutionary process of embedding open source capabilities at societal scale is far from automatic or manifestly benign (see above crypto reference).
A lot of technology, institutions, behaviors still need to be imagined, developed, tested and evangelized. That is the good, roll back your sleeves challenge for current and future generations of digital "social" engineers.
I struggle to see any alternative universe that is not dystopic. Genuinely interested if people have articulated "proprietary" visions that are intellectually honest as to their incentives and morally aligned with ideas of individual agency and freedom and democratic controls.
Compare that with e.g. companies in China or elsewhere that are also working on AI that are perhaps sharing a bit less. That's a bit of a black hole in terms of information going in but not coming out. I think the more open people can get the better. But even some weaker licenses are better than nothing.
Things are open enough currently for people to independently replicate each other's results. I think that's the important bit.
At least for image synthesis, I believe the open models just plainly produce far better results by now since training is crowdsourced.
Then OpenAI and GPT3 was a wakeup call that releasing code to view is not the 'end all'.
Then Facebook and llama, was a final straw of annoyance.
MIT/Apache/GPL or bust. If you use those, you are a person of high class.
They are trying to branch out into training AI models and running inference themselves. But their costs for inferencing look extremely uncompetitive compared to GPU providers like coreweave.
One of the biggest problems Tech companies face is sustainable scaling, I hope you can scale at a manner that allows you to stay relevant without teetering at the cliff's edge.
Do you have long term plans for HF?
Seems quite bandwidth heavy so presumably some sort of monetization will be necessary even if following a fairly open model
The Congress shall have the Power . . . To regulate Commerce with foreign Nations, and among the several States, and with the Indian Tribes;
https://www.bloomberg.com/opinion/articles/2021-01-23/congre...
https://www.cato.org/commentary/its-time-make-congress-great...
They'd also ban all politicking outside of an election season on social media, the news, etc and only allow talk about issues affecting the country and Congress that are under their pervue.
They'd also have better rules about etiquette in Congress and if you defy the rule there's a 5k fee, that goes up 1k every time you repeat. Congress should function a bit like a court room.
Filibusters should've never been knee capped, and they should be focused on laws that really benefit Americans, before corporations, not take money from rich donors, etc. For example if Congress worked, A life saving drugs like insulin would be price regulated. At least after the drug is break even with R and D costs, then allow a max 150 percent markup. Current markup is 1700 percent iirc.
If Congress worked they'd figure out a way that everyone can receive guaranteed healthcare. Every other country does it. They don't want to, because insurance lobbies don't want them too and they pay them their weight in gold.
If Congress worked they'd realize the more money people locally have to spend the more money gets spent. fuck, even Henry Ford understood this concept, essentially it's the reason for the 40 hour work week. Without leisure time he postulated that people wouldn't buy cars because they'd be too worn out. He was right, far sales rocketed.
Middle class and lower class generally spend 90 percent of their income and most locally to the country or their own city. Money doesn't trickle down, it flows up.
Congress is contributing to the polarization of America, you can look at congressional sessions over time and they've evolved to this cesspool that Congress is today, just compare 96 with today or 2006 even.
If Congress worked they'd see the value of ranked choice voting. Even if it cost them their job. Even if it led to many third parties winning elections, hell especially this. The founding fathers knew the two party system would be our ruin but didn't have a plan to tackle this, but they cared about our future. Congress cares about today and maybe 2 years from now max, otherwise global warming would have more focus, though I guess tomorrow is catching up with us on that one.
The AMA/Physician Cartel hates competition, they won't even like Physicians from other countries practice without going through the (regulatory capture) ACGME bottleneck.
LLMs have taken/will overtake a physician's abilities for diagnosis. The math/science will prove that LLMs lead to better outcomes. It seems inevitable.
However, when one person is misdiagnosed(and dies or makes for a sad, emotional story), the cartel will be able to point to it, and outspend everyone else lobbying/bribing to make LLM for medical use illegal... for our safety.