Hacker News Reader

Comment by braingenious | Hacker News Reader

I have not found alpaca to be comparable to chatgpt, but it could be because of bugs in the version I installed through dalai. I might try reinstalling it because I suspect there might be some sort of file corruption issue or whatever.

I gave it the prompt “cats aren’t always fuzzy” and it wrote a lengthy livejournal-esque rambling journal entry about a woman and her husband having money issues. It was funny, but lightyears away from chatgpt.

It does sometimes create some really funny hallucinations though, like inventing prefectures in Japan that don’t exist etc.

View on HN

I also got that text about the married couple and their money issues. Alpaca didn't impress me at all so far.

Alpaca wasn't great. The 13b and 30b models are much better, but just for sentence completion.

Personally, I think that the RLHF does make a big difference but maybe it's a bug in the quantization code as suggested up thread.

I’m also a bit confused by the quantization thing. Why exactly is everybody running the same program on the same file? Why not just include the quantized weights?

It seems like if somebody figured out the “correct” way to quantize the 7b weights it would make way more sense to just torrent the output rather than distribute a fixed program.

quantization takes lots of RAM:

https://github.com/qwopqwop200/GPTQ-for-LLaMa/blob/main/READ... says llama-13B takes 42GB and 33B takes more than 64GB...

I quantised all of the models(though the 65Bn one came out weird) and I only have 64Gb of RAM, so dunno.

There is no reason not to do that, except

i) Distributing large files through torrents is slightly annoying if you don't already happen to have a seedbox

ii) People are still messing around with quantization settings, they might think that they are a few days away from a much better version

iii) No one wants to be sued by Meta. I think the risk is pretty small but not zero.

Reply on news.ycombinator.com