Hacker News Reader

Comment by whimsicalism | Hacker News Reader

It appears there is this genre of articles pretending that LLAMA or its RL-HF tuned variants are somehow even close to an alternative to ChatGPT. Spending more than a few moments interacting even with the larger instruct-tuned variants of these models quickly dispels that idea. Why do these takes around open-source AI remain so popular? What is the driving force?

I've posted this before, but it seems like this genre is just getting more and more popular - and more and more untethered from any actual metrics of how good these models are.

View on HN

They are comparable to the first ChatGPT proof of concept from say early 2022. The reason many of us are excited is because we may be a year or two away from being able to run a ChatGPT if the open source models follow a similar curve.

I have been using ChatGPT from the beginning :) ChatGPT did not have a PoC in early 2022.

They are not comparable with late 2022 ChatGPT.

I assume you mean GPT-3 - ChatGPT didn't launch until November 30th 2022.

Because OpenAI or Google being the gate keepers is a show stopper for most serious people in the space.

They will be able to shutdown your startup on a whim. Even if they didn't politicians and regulators would be a huge risk. Without democratization we get blade runner . Not that democratization has no problems, just that it is the way a lot of us are wanting it to go.

And basing your startup on a model pirated from Meta isn't risky?

I get it, I understand why people like decentralization - but the open source community doesn't even have close to the capability to train an actually open-source LLaMA equivalent.

Mozilla or Wikimedia foundation have budgets with the right number of zeroes to support an effort like this.

While those organizations might not be the right fit, I think they serve as an "existence proof" that a large scale open / nonprofit project is not completely inconceivable.

Could possibly see an industry consortium (of players too small to compete on their own) funding an open effort.

Last idea sounds crazy but hear me out: how much would Nvidia spending $100M on "open" models boost spending on graphics cards? I hope someone's running the numbers on that...

> how much would Nvidia spending $100M on "open" models boost spending on graphics cards?

Probably not as much as in a zero-sum game where everyone is trying to train their own model. Every leap in CPU inference makes this an increasingly less appealing option for them.

But I agree, some industry consortium might try to do it. I think they would first have to be relying heavily on LLMs before they were willing to do that, and its possible by then that the lead will have gotten too large to easily surmount, especially now that everyone has stopped publishing.

You make it sound as if ChatGPT itself isnt severly limited. Vicuna and newer 13B models are quite close . And the uncensored models have a capability that ChatGPT will never have.

They just aren't quite close. Maybe you guys are asking shallow questions, but I'm asking subject matter questions and this is just not true.

It would be great if we could have detailed QA evaluation to show this, but of course then the open source people would train their models on it as a fine-tuning datasaet.

Train it on specific subject matter and not waste space on things like “what year did Jack Nicholson beat on that dude’s car with a golf club?”

I mean… the horror.

That's just not how these models work.

You’ll have to forgive them, they’re probably trained mostly on HN posts. ;)

What are the newer models? I am testing them in batteries across complex tasks, so far vicuña is the most flexible but they all choke on reflective instructions (I. E. Knowledge to extract is not in the model but in the user text)

I very interested in this usecase - where should I go on the interwebs?

original paper: https://arxiv.org/abs/2205.00445

langchain agents are a good starting implementation.

you can build your own prompt and get the ai to work by iself hallucinating tools, which may be cheaper to test out than going back and forth with an agent manager. not as accurate, but you can still extract useful work, i.e. https://i.imgur.com/AE4R3dR.png (gpt-35-turbo is traditionally failing this task completely, prompt get it to work at it)

these prompt all require the model to work off data within the prompt within the first shoot. model require a degree to introspection for that to work.

The way OpenAI has set their models up for prompting and follow-up is where their real advantage is. It's very hard to take a model you can run locally and just go "Hey, write me a powershell script to convert all these files into this format. Split them up so they're no more than an hour each. Ok, change it to also denoise them."

Are you talking about the RLHF? That's where Vicuna/Alpaca and other finetuning help a lot.

There's also already tools for conversation flows (which just means you prepend the conversation history to the prompt).

I'm not saying the performance is nearly as good, but the actual workflow does already exist and is massively improving. The interesting part to me is that this finetuning can be done in a couple few hours on a consumer gpu (4090).

People are enthusiastic about the possibilities, imagine a black box on your desk contains only RAM and matrix multiplication chips, you install on it your favorite AI assistant and you train it with your private data/code, you remove all prudish restrictions and get productive on your work and on your off times.

Llama has the potential to reach ChatGPT it needs tunning to get better at responding to questions, llama if I am not worng is mostly attempting to predict what is next.

I can see it similar like Midjourney and Stable diffusion, midjorny can make any stupid prompt look like a digiatal art in the style of Midjorney but look how many stable Diffusion innovation happens, a competent person that is on top with all the new stuff can produce absolute anything in any style they want.

I am similarly enthusiastic and super, super impressed by Llama.cpp

But I just don't see the need to mislead and suggest that these models are "on par" with ChatGPT or something like that. They just aren't.

I did not said that are the same, the llama has a lot of information in it , the issue seems to be the Q&A chat part. This can be added on top by the community without having to start from scratch IMO. But I might be wrong, and OpenAI put some magic shit in , some super secret unpublished stuff, in that case educate me. In my mind I compare with Stable Diffusion, the proprietary ones produce more artistic effects because they put some more stuff into the prompts and they are imposing their styles. With SD you have the control and you need to wait for some improvements, plugins, new loras or embedings with new styles etc. In SD I could train my face in 1- minutes, with proprietary shit it will never happen...

Imagine what a math community could train, they just need access to the model and soem GUI software that can help them train.

So llama based Chat stuff is not yet comparable with ChatGPT but there are already lot of progress made. At this moment coding and math is bad in llama based but other stuff is great, like story creation, also I only could test 3-b 4bit and it is good enough to for example provide me a complex response in valid JSON format.

My hunch is model weights will be commercialized as a purchased object. They can be watermarked so easy to trace any leak.

Then hardware will be a separate business. I think Apple might be caught off-guard by Nvidia on hardware. The latest NVlink and 400Gbps interconnects when combined with H100 next iterations and also rumors of advanced PCIe motherboards with high lane Nvidia CPUs and it looks to me that next year they can be selling $100-300K physical systems optimized for LLM inference that physically remind me of mainframes.

It makes no sense, the science is free and open, the companies just put their money into throwing data into the model. Once you have the big llama model filled with all the humanity information as an open source thing at bast a company could sell you some small stuff to add on top, like maybe Disney would sell you a "license" and a lora to generate Disney crap, their model would probably will be lower in quality then the open ones but the license would be the important part.

It is kind of idiotic that some scientist can spend years and a lot of public money to create some technology and then bilionairs are miliking all the profits.

I definitely hope you are correct.

I’m really hoping there are viable distributed and somewhat decentralized eventually consistent training algorithms we could all run in a P2P system. That would be super cool.

However I can easily see that now the framework has been established if a company builds a proprietary curated dataset for specific skills and then pays to spend resources on specialized reinforcement training.

Then they can commercialize that I would think. As people would pay for an LLM that does XYZ the best. Kinda like your Disney example but I was thinking engineering tasks in my head.

I'm interested as LLMs are interesting tech but ChatGPT is proprietary and useless for free software.

Agreed. I’ve said it elsewhere: how do we know that ClosedAI has fully published all their tricks used to build GPT3.* or later? The main “clues” people have are from the InstructGPT paper, but it should shock nobody if it turns out that paper reveals less than 10% of their techniques, which may have taken years to come up with. Competitors would need to rediscover those on their own or come up with new ideas. Simply repeating the ideas from that paper is likely not going to get them to a competitive model performance.

Reply on news.ycombinator.com