Comment by jstarfish | Hacker News Reader

Comment by jstarfish | Hacker News Reader

Parent Full thread

The government's interests are going to align more with OpenAI's, and they would be fools not to eliminate open source/local models.

Anything local bypasses traversal of any network edges they can tap and monitor. Anything you ask Google is logged, and your search history can be subpoenaed, same as with any public library and the books you check out. OpenAI or whatever gatekeeper would assume that role and charge tolls.

Do the same research with Vicuna-13B-ggml or whatever and nobody can sniff out what you're doing. Which sounds great for people who just want to jerk off and be left alone, but has terrifying implications for anybody tasked with stopping more-malicious users.

Unfortunately nobody can ever compromise on these things. Best case would be adopting a law like the recent one in NL, where given probable cause the government can hack into your machine, but that won't happen. The only alternative in their favor is banning local models and open-source development of them altogether.

That worked well in the past with crypto. /s

People yearn for "a room of their own"[1], a sanctuary for contemplation and creativity. There was a time when our minds and pieces of paper were the only canvases for our imaginations, but increasingly our web browsers have morphed into thought surveillance systems. Now, the tide is turning back towards private creative spaces, enhanced by generative AI. Even if there are attempts to ban it, the edge now lies with private AI. In the future, we might even have the ability to run even more powerful models on standard hardware.

Moreover, we must consider the defensive strategies against other AI entities that will soon permeate the internet. Operating much like a firewall, private AI models could safeguard users from external threats such as involuntary data breaches and misinformation. This will be an endless cat-and-mouse game akin to battling security exploits or viruses—there's no one-size-fits-all solution. Venturing into the internet without your AI shield may soon be as unthinkable as stepping outside without a mask amidst a pandemic like COVID.

[1] https://en.wikipedia.org/wiki/A_Room_of_One%27s_Own

99% of people with crypto own it through a middle man.

edit: you're talking about crypto in general. Bitcoin spam has broken my brain.

I can run GPT-3.5 level models on my laptop (albeit a chunky high end one). It is indeed only a matter of time.

I’ve unfortunately not found an open source model that can perform at the this level for anything beyond madlibs-style text generation. Most of my prompts are a combination of sequential tasks and one-shot prompting, typically between 1K-2K tokens and everything I’ve tried falls down at this. Granted, I haven’t gone beyond 30B models, but the gulf still seems quite wide (and even wider compared to GPT-4). Really hope it does close in the next year!

I admit I have not pushed them that far, though I am also using 40B and 65B models which may perform better than a 30B model.

Nothing open approaches GPT-4 yet, which I have heard is an overlay of multiple 220B models.

The only way you will see that on even high end consumer hardware is advances in compression and pruning, but those are happening. There’s a lot of research in those areas.

Of course then cloud models could use the same techniques to get even bigger, if the model and training will keep scaling.

Edit: as AI matures you may also see lots of custom ASICs hit the market. I could see those getting cheap enough to run many hundred B models locally. The acceleration of an ASIC designed for a specific type of model over CPU and GPU could be dramatic, reducing the amount of silicon you need.

In my experience, Wizard-Vicuna output at 13b will match or exceed the prose quality of GPT3.5 in smaller generation tasks. I've been using it for a Discord bot to great success.

There is an argument about the open source models causing more innovation and thus more industry and/or competetive advantage of a nation. So I can't imagine that control is the only thing they are thinking about, though surely they are thinking about control.

Reply on news.ycombinator.com