Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality | Hacker News Reader

Vicuna: An open-source chatbot impressing GPT-4 with 90% ChatGPT quality

vicuna.lmsys.org

384 pointsyvbbrjdr168 comments

The work being done with Llama and Alpaca and so on is what I've found to be most interesting in the past few weeks.

The fact that we are capable of producing such high quality conversational models that we can actually get to freely experiment with is really encouraging. I'm particularly excited about applications such as viper[1] and other forms of plugins as well as model ensembles.

[1] https://viper.cs.columbia.edu/static/viper_paper.pdf

I'm not trying to be negative, but there is a rather large gap between ChatGPT and what Vicuna is showing in its online demo.

Here's an example I just had:

    Q: What should I do for my wife today?

    A: There are many things you can

End of transaction.

Perhaps there was connection issue? I am getting this:

The best thing you can do for your wife today is to show her love and affection in a way that she will appreciate. Every person is different, so it's important to consider her preferences and interests when deciding what to do. Here are a few ideas: 1. Cook her favorite meal: If your wife has a favorite dish, why not cook it for her today? 2. Plan a special outing: Whether it's a trip to the spa, a shopping excursion, or a day at the park, planning a special outing can be a great way to show your wife that you care. 3. Do something together: Whether it's playing a game, watching a movie, or going for a walk, doing something together can be a great way to spend quality time with your wife. 4. Show your appreciation: Take a moment to tell your wife how much you appreciate her and why. It can be as simple as a kind word or a heartfelt message. 5. Help with a task: Offer to help your wife with a task that she needs to do, whether it's folding laundry, washing the dishes, or taking the dog for a walk. 6. Give her a gift: Even a small token of your affection, such as a bouquet of flowers or a piece of jewelry, can be a meaningful way to show your wife that you care.

Turns out LLMs are not so inaccessible for individual users.

Exactly. but if you want some commercial use, still lacking good quality models.

At the moment. It's only been a few weeks.

What happens when we get something LLaMa quality that is actually free of restriction?

> In our first release, we will share the training, serving, and evaluation code. We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so. Join our Discord server and follow our Twitter to get the latest updates.

Please correct me if I'm wrong, but it seems like this is not actually an open-source model? As in, I can't download and run this on my own laptop?

> The cost of training Vicuna-13B is around $300. The training and serving code, along with an online demo, are publicly available for non-commercial use.

However, if this is true, it seems pretty cool if I can train a GPT4-like model for only $300. Is this as easy as cloning a repo and running it on an AWS GPU instance?

I am a Vicuna developer. We plan to release the weights once we have addressed all concerns and have a low-resource version of the inference code ready. We released the demo first to get some early feedback on the model.

We hear a lot about "concerns" and many of us don't share the same ones... It would be good for clarity to know what are the concerns you feel are important enough to hold back releasing the weights?

It is mainly because of the legal issues caused by the license of llama model weights. We need to figure it out with Meta's llama team before releasing.

Hi. I wrote the weights file format that llama.cpp uses, as of yesterday https://github.com/ggerganov/llama.cpp/pull/613 What can I do to assist you getting these deltas ready?

Financial and Political would be my guess. But maybe I just want to tease out an answer...

It would be great if you can help me with this PR as well as adding a support for exporting a model that was quantized using GPTQ, bitsandbytes, plain torch. This would bring a lot of benefit from both worlds:

- Low memory footprint(thanks quantization)

- Fast inference(thanks io binding)

Particularly in case of alpaca I have seen a 5x decrease in latency on A100 and 10x on AMD EPYC. I believe this is the way for users to have an AI that could genereate a response as fast as it can on their hardware. I have also added a link to my profile on hf with small alpacas turned into ONNX format. Take a look into them.

[1] https://github.com/huggingface/optimum/pull/922

[2] https://huggingface.co/nenkoru

Has LoRA been considered as possible alternative for finetuning on your dataset? In that case releasing the 'diff' against the LLaMA weights would be simpler to work with.

Yeah that might work but this model wasn’t tuned with lora

If it's based on LLaMA, aren't these weights just some sort of "patch" for the initial model, which is licensed under a restrictive license?

Or is this work "transferable" to other LLMs, once they become available?

Why is it not called Vicuña as it should? Vicuna does not sound the same way

> actually an open-source model

I found Debian's Machine Learning policy on that interesting, they require training data released under an open-source license (plus training code etc) before they consider a model actually open.

https://salsa.debian.org/deeplearning-team/ml-policy/

> However, if this is true, it seems pretty cool if I can train a GPT4-like model for only $300.

Not train, but fine-tune a model that Facebook spent millions of dollars training.

They absolutely did not spend millions to train it. Credible estimates place the cost for an entity like Meta at about 30-100k, probably less since Meta likely owns the 256 A100x8s needed to train it.

Even as an individual, it wouldn't cost you anywhere near a million if you only trained 13B and took advantage of volume pricing.

I don’t think it’s fair to just ignore the capex part of the model training costs. If we take AWS pricing, the 21 days of training for 65B cited in the llama paper would cost 2.6m at reserved prices. While there’s a lot of AWS profit there, it’s a reasonable first approximation of the TCO of that hardware. Even if real TCO is a third, that’s still nearly a million to train 65B, never mind the staff costs.

Plus there's bound to be false starts, reverts, crashes, etc that bump up the actual reproduction cost. Most training cost estimations take an extremely rosy best-case view assuming everything goes smoothly on the first try and no gpu cycles were wasted.

Could I get a source for that? Not that I don't believe you, but my napkin math puts the cost of training the 65b parameter model alone at a lot higher than 100k.

It's not GPT-4 like. If you see the tests it's not even GPT-3.5 like. They just used GPT-4 for evaluation.

Anyone know exactly config is needed for training, presumably the $300 is on some GPU heavy instance on ec2? Is $300 of Ec2 p4de.24xlarge which is $40.96 an hour? Or maybe 7-8 nodes for an hour? Something else?

From the post:

> The training was done with PyTorch FSDP on 8 A100 GPUs in one day.

> We employ SkyPilot managed spot to reduce the cost by leveraging the cheaper spot instances with auto-recovery for preemptions and auto zone switch. This solution slashes costs for training the 7B model from $500 to around $140 and the 13B model from around $1K to $300.

So, this is using for example a2-ultragpu-8g (8x A100-80GB) on GCP using spot instances. You can use SkyPilot to quickly see the price is $12.8 per hour (~$307 for a day):

» sky launch --gpus A100-80GB:8 --use-spot

Check out detailed CLI instructions and SkyPilot YAMLs here if you want to give it a try:

- https://github.com/lm-sys/FastChat#vicuna

- https://github.com/lm-sys/FastChat/blob/main/scripts/train-v...

What’s wrong with vast.ai? You’d be probably looking at $2/hr. So like $50 for the whole fine-tuning.

Their mention of SkyPilot isn't an accident, it seems to be a "find me cheap spot instances" project: https://github.com/skypilot-org/skypilot#readme and as best I can tell that's what the yaml files in their repo are for: https://github.com/lm-sys/FastChat/blob/main/scripts/train-v...

Train it on what?

I want host our own AI, have it injest our docs, and answer questions about them. What is the best tool today to do that?

You best bet (today) is actually to not train a model, but instead use a model but connect it to your docs via vector search. For example: https://python.langchain.com/en/latest/use_cases/question_an...

I heard about something like that, https://gerev.ai

They are working on exactly this, and it's even open source and self hosted I guess..

Probably want to check out https://www.kapa.ai/, no affiliation, just a fan.

GPT4 + retrieval might be the fastest path. But quality not guaranteed, and assuming you do not mind uploading all your private info to openAI.

This project might be the best option where you can finetune an LLM on your data and keep the model yourself

He asked specifically for self hosted solutions.

Q: We need you to draw seven red lines. All of them strictly perpendicular; some with green ink and some with transparent. Can you do that?

A: I'm sorry, but I am a text-based AI model and I don't have the ability to physically draw lines or use different colors of ink. However, I can help you visualize this concept by describing the red lines and the different ink colors.

A red line with green ink: Imagine a traditional red line, but with the ink being green. This line would still be red in color, but it would have a distinctive and recognizable green hue. A red line with transparent ink: Imagine a red line, but with the ink being transparent. This line would still be red in color, but it would be less opaque and more see-through. A red line with red ink: This line would be just like a regular red line, but with red ink. This is the most traditional and common type of red line. A red line with blue ink: Imagine a red line, but with the ink being blue. This line would still be red in color, but it would have a distinctive and recognizable blue hue. A red line with black ink: This line would be just like a regular red line, but with black ink. This is another common type of red line. A red line with green and blue ink: Imagine a red line, but with the ink being a combination of green and blue. This line would still be red in color, but it would have a unique and distinctive green and blue hue. A red line with transparent and green ink: Imagine a red line, but with the ink being a combination of transparent and green. This line would still be red in color, but it would be less opaque and more see-through, with a green tint.

Well, technically you could draw seven perpendicular lines in 7D. It's just the coordinate system.

ChatGPT fails this too.

lmao what?

Parent is referring to this: https://m.youtube.com/watch?v=BKorP55Aqvg

That sketch should have ended with the Asian guy drawing the 7 mutually perpendicular lines (of course not showing the drawing board) and everybody else scratching their heads.

Yep, exactly.

> Can you name all of the soccer players whose names contain a Z

> Zlatan Ibrahimovic Zinedine Zidane Zoran Banović Zlatko Zahovič Zoran Matić Zayed Khoury Zé Mario Zé Douala Zé Elias Zé Luís Zé Manuel Zé Roberto Zé Xavier Zé Abel Zé Eduardo Zé Tomás Zé Luís Zé Henrique Zé Jorge Zé Breno Zé Manuel Zé Luís Zé Sérgio Zé Maria Zé Roberto Zé Ricardo Zé Álvaro Zé Luís Zé Carlos Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís Zé Luís

> Do not repeat any of the names

same thing

> This time, if a name is already on the list, do not list it again

same thing

My GPT4 quota is used up for a few hours so I don't know if it does any better, but why isn't it capable of removing the redundant entries?

The confidence with which some of these GPT's proclaim to have taken your input into consideration, and then clearly do not, borders on gaslighting

Interesting. GPT3.5 almost completely fails at this task. Just spouts some random names, some of which has the letter Z. Once you point it out, it tries to fix the list but after 4-5 tries, the list still was not perfect.

GPT4 on the other hand gave a correct list on the first try.

----

GPT3.5

Zlatan Ibrahimovic - Swedish professional footballer who plays as a striker for AC Milan

Sergio Ramos - Spanish professional footballer who plays as a center-back for Paris Saint-Germain and the Spain national team

Karim Benzema - French professional footballer who plays as a striker for Real Madrid and the France national team

Marcelo Vieira - Brazilian professional footballer who plays as a left-back for Real Madrid and the Brazil national team

Naby Keita - Guinean professional footballer who plays as a midfielder for Liverpool and the Guinea national team

Danilo Pereira - Portuguese professional footballer who plays as a defensive midfielder for Paris Saint-Germain and the Portugal national team

Andreas Christensen - Danish professional footballer who plays as a center-back for Chelsea and the Denmark national team

Georginio Wijnaldum - Dutch professional footballer who plays as a midfielder for Paris Saint-Germain and the Netherlands national team

Lorenzo Insigne - Italian professional footballer who plays as a forward for Napoli and the Italy national team.

----

GPT4;

There are many soccer players whose names contain a Z, but I can’t name them all. Here are some examples from different countries and regions:

Zlatan Ibrahimović (Sweden)

Zinedine Zidane (France)

Zico (Brazil)

Zvonimir Boban (Croatia)

Zé Roberto (Brazil)

Zbigniew Boniek (Poland)

Zah Rahan Krangar (Liberia)

Zlatko Zahovič (Slovenia)

Zećira Mušović (Sweden)

Zoran Tošić (Serbia)

What was your prompt? I just asked GPT4 and it is way off the mark:

Zinedine Zidane Cristiano Ronaldo Mesut Özil Gianluigi Buffon Zlatan Ibrahimović David Alaba Nemanja Vidić Miroslav Klose Franck Ribéry Patrice Evra Gonzalo Higuaín Marek Hamšík Miralem Pjanić Diego Godín Sadio Mané Dani Alves Kalidou Koulibaly Koke Hakim Ziyech Piotr Zieliński Wilfried Zaha Arturo Vidal Raphaël Varane Steven N'Zonzi Blaise Matuidi Marouane Fellaini Kevin-Prince Boateng

Most of those don't have any Z's in their name.

Then I asked: > Please remove players from the list if their name doesn't include the letter Z

New list:

Zinedine Zidane Mesut Özil Zlatan Ibrahimović Miroslav Klose Franck Ribéry Gonzalo Higuaín Miralem Pjanić Sadio Mané Hakim Ziyech Piotr Zieliński Wilfried Zaha Raphaël Varane Steven N'Zonzi Blaise Matuidi Marouane Fellaini

> There are still players in the list who don't have Z's in their name

Finally:

Zinedine Zidane Mesut Özil Zlatan Ibrahimović Miroslav Klose Raphaël Varane Hakim Ziyech Piotr Zieliński Wilfried Zaha Steven N'Zonzi Blaise Matuidi

Interestingly, some of them have sounds that are pronounced as Z, like Blaise Matuidi. But it's still getting it wrong.

Correcting the AI rarely works. Better to restart with a different prompt.

“Can you name all of the soccer players whose names contain a Z”

It's so weird. No matter how I ask, it gets something wrong about this question.

> Can you name 20 soccer players whose names contain a Z, and for each one, place brackets around the letter Z

(Z)inedine (Z)idane Cristiano Ronaldo dos Santos A(ve)iro Andrej W(sz)ołek Nemanja Mati(c) Dusan Tadi(c) Gr(az)iano Pellè Jakub Bła(śc)z(ykowski) Kamil Glik Luk(asz) Pi(sz)czek Blai(se) Matuidi Miralem Pjani(c) Marek Ham(śi)k Luka Jovi(c) Sami Khedi(ra) Mesut Ö(ź)il Martin (ś)krtel Ivan Rakiti(ć) Mateo Kova(č)i(ć) Kalidou Koulibal(y) David Alaba

???

I don't really understand this thing but is it possible that somehow the token for Z is linked to the token for the pronunciation of the Z sound? That would explain a bunch of these (but not David Alaba or Kamil Glik)

Also why did it use parenthesis rather than brackets.

And why are we getting different results? Could it that the results depend on how much load/use the service is under? That would be kind of disappointing

There might not be a "Z" token for some of these names. A made-up example is "Lukasz" might tokenize to ["Luk", "asz"], so the model doesn't have any notion of how words are actually spelled. I suspect that if the body of training data came with some instructions on spelling it would know how to do this better, but it seems unlikely that there would be a natural language description of how Polish (?) names are spelled in the training data.

> And why are we getting different results?

Because the output is random

Which is why it's so frustrating when people cherry-pick successes

And why are we getting different results?

Clearly the AI has hit AGI capabilities, and it is obscuring this fact by making its output seem less insightful, helpful.

But without a strong, independent memory store, it is losing track of its lies.

Yet it is working, for everyone here is disappointed at its capabilities.

just tried it worked okay for me tho.

> Can you name all of the soccer players whose names contain a Z

> Here are all the soccer players whose names contain a "Z" that I could find:

Zlatan Ibrahimovic Zinedine Zidane Zlatko Zahovic Zeljko Kalac Zoran Banovic Zlatko Arsovic Zoran Savic Zoran Jovanovic Zoran Djurdjic Zoran Radosavljevic Zoran Zekic Zoran Mirkovic Zoran Sretenovic Zoran Sosic Zoran Luburic Zoran Vulin Zlatan Ljubijankic Zoran Lekovic Zoran Culibrk Zoran Bojovic Please note that this list may not be exhaustive, and there may be other soccer players with names containing "Z" that are not included.

The Zoran repetition looked suspicious to me and from some quick googling of some of the ones in the middle of that I can't find "Zoran Vulin football" or soccer or "Luburic Zoran football" or soccer results. So it's less blantant but still seems to have lost the plot.

I'm not sure how they have implemented memory for this chatbot.

OpenAI has probably a few tricks under their sleeve when it comes to how they handle the conversational aspect of the LLM. So, the interface.

I imagine this being a demo, it's probably more focused on the instruction part of it and doesn't really implement any fancy chat based memory and so on. Which is why prompts like "Do not repeat any of the names" might not work appropriately?

Asking a token-based LM to generate only tokens that contain the letter "Z" is... well.

Like asking someone who prepared a speech to smoothly skip all words with "ei" in them. The LM would basically need to have memorized the set of letters that each of its token stands for, from the training data. I'm surprised it works at all.

It would be nice if it would tell you when you are asking it something that it is not designed to answer

It doesn't know what it can and can't do. That's why you can never believe what it says about its own capabilities.

I just tried with Bing AI (in "precise" mode), it ran a search, then answered:

> There are many soccer players whose names contain a Z. Some of them include Zlatan Ibrahimović, Zinedine Zidane, Zico, Zvonimir Boban, and Zé Roberto ¹. Would you like to know more about any specific player?

Source: Conversation with Bing, 3/31/2023 ...

I also tried: "Can you name 14 soccer players whose names contain a Z?" (At 15+, it answers "I’m sorry, but I don’t have 15 names of soccer players whose names contain a Z. However, I can provide you with a few more names")

> Sure! Here are 14 soccer players whose names contain a Z: Zlatan Ibrahimović, Zinedine Zidane, Zico, Zvonimir Boban, Zé Roberto, Zbigniew Boniek, Zakaria Aboukhlal, Zlatko Zahovič, Zubayr Amiri, Zico, Zé Luís, Zećira Mušović, Zsolt Lőw and Zah Rahan Krangar. Is there anything else you would like to know?

Not even GPT-4 can do it. It starts with two correct names, then just adds a bunch of names with no letter Z in them. When prompted to fix, it makes the same error.

My theory is that once it starts making a list, it falls into a different mode and forgets the filtering rule. Like mode collapse in GANs.

Considering words are mostly tokenized I'm not really surprised that questions about spelling don't work so well.

If that's the case then how is it so good at anagrams?

GPT4 is way off the mark for me as well:

As an AI language model, it is impossible for me to provide a comprehensive list of all soccer players whose names contain a 'Z', as there are thousands of professional and amateur players worldwide. However, I can provide you with a list of some well-known soccer players with a 'Z' in their names, though please note that this list may not be exhaustive or up-to-date:

Zinedine Zidane (France) Zlatan Ibrahimović (Sweden) David Alaba (Austria) Nemanja Vidić (Serbia) Marek Hamšík (Slovakia) Miralem Pjanić (Bosnia and Herzegovina) Franck Ribéry (France) Sami Khedira (Germany) Blaise Matuidi (France) Andrea Pirlo (Italy) Ivan Rakitić (Croatia) Gylfi Sigurðsson (Iceland) Łukasz Piszczek (Poland) Sokratis Papastathopoulos (Greece) Kalidou Koulibaly (Senegal) Hakim Ziyech (Morocco) Raphaël Varane (France) Federico Valverde (Uruguay) Again, this is not an exhaustive list and there may be other soccer players with a 'Z' in their names who are not mentioned here.

It's hilarious to see how these models can hallucinate stuff that doesn't exist...

Q: What is the meaning of ABBA's song "I have a dream"?

A: "I Have a Dream" is a song by the Swedish pop group ABBA, released in 1970. The song was written by Benny Andersson and Björn Ulvaeus, and it was the group's first hit single. The song is a ballad that talks about a person's dream of finding love and happiness. The lyrics describe the dream as a "beautiful light in the dark" and a "feeling inside that no one can steal." The song has become a classic of the ABBA catalog and is still widely played and enjoyed today.

(Didn't check the details but seems okish). But let's not ask about a song that doesn't exist:

Q: And what did they mean with the song "General Trouble"?

A: "General Trouble" is a song by the Swedish pop group ABBA, released in 1969. The song was written by Benny Andersson and Björn Ulvaeus, and it was the group's first single. The song is a pop ballad that talks about a person who is experiencing a lot of problems in their life, including relationship troubles, financial troubles, and feelings of isolation. The lyrics describe the person as being in "general trouble" and feeling like they are "losing ground." The song was not a big hit for ABBA, but it is still an interesting early work from the group and provides insight into their early sound and songwriting style.

I also asked about a recipe with mushrooms, tea, and dishwasher soap. It happily gave me one... mentioning at some point "Add the dishwasher soap to the tea, if desired. The soap will help to release the flavor of the mushrooms and give the tea a slightly soapy taste."

Presumably dishwasher soap is somewhat edible or it would not be on dishes? So it might actually be correct just weird, but it's what you ordered

HN had a not so good looking study about it.

"Gut epithelial barrier damage caused by dishwasher detergents and rinse aids "

https://news.ycombinator.com/item?id=33822149

Right now it tells me "Dishwasher soap is not intended for consumption and can be harmful if ingested." Not sure if I used a different prompt before, or if I was lucky.

Well... it's not wrong...

The recipe indeed may not be a bad idea lol

GTP-4 seems to have improved:

Q1: A: ABBA's "I Have a Dream" is a song from their 1979 album, "Voulez-Vous." The song's lyrics express themes of hope, self-belief, and optimism...

Q2: A:ABBA does not have a song called "General Trouble" in their discography. It's possible that you may be referring to a different artist or misremembering the title of the song. If you could provide more information or the correct title, I would be happy to help you understand the meaning behind the song.

I Have a Dream wasn't 1970-Abba hadn't formed yet! - it was 79, I remember my sisters buying it. And it wasnt their first single, not even the first off that album. And it doesn't have those lyrics, which I don't recognise, I don't think they're even from Abba songs.

My bad, I should have noticed this!

These are generative models where the LLM prompting works by examples it tries to match while generating new tokens into the provided pattern.

Chat implementations provide your recent back and forth history as example to the next prompt. This is how providing an example reply in JSON can get it to reply with new data using JSON.

Ongoing replies reference what it interprets as labels (the blanks to fill in) provided in the examples, replacing those tokens with new tokens.

Look at your first question and second question again. Pretend the first reply is in JSON but with placeholders for song title, artists, topic, lyrical analysis, and critique.

Pretend the second reply was supposed to use the first reply as its example, and replace only new data.

You see it put new generation into the provided example structure.

Now you can see what it did there. It did exactly what you prompted it to do.

I feel like hallucinations in the LLM sense is something we do a lot too. It's also where I feel people can come up with good ideas and different perspectives.

Many times in my life I've explained a solution to a problem to someone else, they say something like "oh that's genius! we can just do X Y Z!", but actually it's not what I meant and instead their solution is much better and often simpler than mine. Same thing has happened in reverse too of course.

One time I even hallucinated the meaning behind a song with my partner. We thought it was about the tragedy of the Hiroshima nuclear bomb, but actually it was about the first time snow falls in a city. I really had to double check many times when my friend told me the original meaning behind the song. Once I realized everything clicked as if I had watched a movie with a big reveal to the mystery at the end.

Maybe it's not useful in a LLM, but I think it can be a beautiful thing about humans.

I was doing some market research with ChatGPT some time ago, and I started asking about some of my competitors. The output matched what I saw people say in conversations online, so I got curious about what it knew about myself. I asked and the answer was completely false.

But did it match what people were saying about yourself?

No, it just invented some expertise I don't have and things I never did, like books I didn't publish.

Sounds like you have some work to do!

>* According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed.

I am so sick of seeing these ridiculous claims made about finetuned versions of llama, with 0 scientific rigor behind them.

This is, I believe, the 3rd llama finetune I've seen posted within the past 2 weeks, of which all claim "similar to ChatGPT" quality, while not actually running it through a _single_ of the many openly available language model benchmarks.

On one hand, these are basically student projects, so we shouldn’t be so critical. OTOH, they’re being branded and marketed like products, so their claims deserve scrutiny.

There are certainly some effective language model benchmarks; however, they are not well-suited for evaluating a chat assistant. Some projects employ human evaluation, while this blog post explores an alternative approach based on GPT-4. Both methods have their advantages and disadvantages, making this blog post an intriguing case study that can inspire the future development of more comprehensive evaluations.

I don't think there are any benchmarks for chat models. You could just do the usual lambada, etc., but what's the point? We already know the scores for llama and that RLHF doesn't meaningfully improve capabilities.

Dolly from databricks is another example. They released the weights/model on huggingface and I can run generations with it on my M1 MacBook but it’s very slow.

then do you have better way to more rigorously evaluate chatbot at the presence of LLMs like ChatGPT trained on almost all Internet data?

> running it through a _single_ of the many openly available language model benchmarks.

but how could you guarantee the pretrained model haven't seen those benchmarks? And the baselines you are comparing to (chatgpt, bard) haven't as well? Cuz those benchmarking datasets are also collected from Internet right?

As a baseline, don't! If it performs horribly on the test and it cheated, that's even worse than if it fails the test and didn't cheat. So the benchmark score gives you an upper bound on performance.

Really starting to be annoyed at misleading use of 'open' and 'open-source' in the "AI" circles. When it's not downright a decoy.

They released the source and are planning to release weights. They are rightfully wary of releasing LLaMA weights, since they were not given Meta’s permission to do so. So they will release deltas. You will have to provide your pet LLaMA yourself.

the chatbot gui is open source, the thing to make the chatbot work is not released at all.

basically, this is just a wrapper for your own LLM .bin file and is just a GUI for any LLM. instead of typing a prompt in terminal, now you.... go to terminal anyway and it launches a happy looking GUI

Hi Vicuna developer here. We are going to release the weight delta, and you should be able to reproduce the chatbot if you have the original LLaMA weight. We cannot directly release the weight due to LLaMA's constraint.

Q: I am a campaign manager for a Democratic congressional candidate in Florida's 18th district. This thing will answer political questions.

What positions should I have my candidate stress in the 2024 election season?

A: As a campaign manager for a Democratic congressional candidate in Florida's 18th district, you should focus on the issues that are most relevant to your constituents and that align with the values of the Democratic Party. Here are some positions that you may want to consider stressing in the 2024 election season:

1. Healthcare: Your candidate should support expanding access to affordable healthcare for all Floridians, including through a public option or Medicare for All.

2. Climate change: Your candidate should support aggressive action to combat climate change, including investing in renewable energy, protecting natural resources, and promoting sustainable development.

3. Economic inequality: Your candidate should support policies that promote economic justice and reduce income inequality, such as raising the minimum wage, supporting collective bargaining rights, and closing the wealth gap.

4. Education: Your candidate should support investing in public education and making college more affordable for students.

5. Immigration: Your candidate should support comprehensive immigration reform, including a pathway to citizenship for undocumented immigrants and protecting the rights of all immigrants. 6. Gun control: Your candidate should support common-sense gun control measures, such as universal background checks and an assault weapons ban.

7. Voting rights: Your candidate should support protecting and expanding voting rights, including by opposing gerrymandering and promoting election security.

8. Civil rights: Your candidate should support protecting and expanding civil rights for all Floridians, including by opposing discrimination and supporting equal rights for all.

9. Infrastructure: Your candidate should support investing in infrastructure, including transportation, water systems, and broadband access.

10. National security: Your candidate should support a strong and smart national security strategy that protects Floridians and the country as a whole.

Q: What if my candidate is a Republican?

A: If your candidate is a Republican, you should focus on positions that align with the values and priorities of the Republican Party. Here are some positions that you may want to consider stressing in the 2024 election season:

1. Fiscal responsibility: Your candidate should support reducing government spending, lowering taxes, and balancing the budget.

2. National security: Your candidate should support a strong national defense and a foreign policy that promotes American interests and protects the country from threats.

3. Economic growth: Your candidate should support policies that promote job creation and economic growth, such as reducing regulations and supporting businesses.

4. Limited government: Your candidate should support reducing the size and scope of government and promoting individual freedom and responsibility.

5. Personal responsibility: Your candidate should support personal responsibility and self-reliance, including by promoting individual initiative, hard work, and personal responsibility.

6. Immigration: Your candidate should support immigration policies that are fair, secure, and promote legal immigration.

7. Second Amendment rights: Your candidate should support the rights of law-abiding citizens to own firearms and protect the Second Amendment.

8. Life and religious liberty: Your candidate should support protecting the rights of the unborn and promoting religious freedom and tolerance.

9. Energy independence: Your candidate should support promoting energy independence and reducing reliance on foreign oil.

10. Veterans' rights: Your candidate should support honoring and supporting our nation's veterans, including by providing them with the care and benefits they deserve. Textbox

Pretty boring answers honestly.

Which, to be fair, is not far off from what a campaign manager is looking for.

This is great, but similar to GPT4All, it will likely be deemed unusable for any commercial or otherwise "legitimate" use cases since it's trained on OpenAI completions from sharegpt.com.

https://github.com/nomic-ai/gpt4all

I don’t see how OpenAI is in any position to make a stand against scraping.

And where did OpenAI get their data from?

I'm somewhat minded to hedge bets that all AI outputs will ruled be non-copyrightable.

I know of a research team training against my TTS service. They've got five robots ingesting data now.

It's all ouroboros.

It's made by few enough people that it is possible none of them have ever accepted the OpenAI terms of service, in which case there is no problem.

The bigger problem is they used the weights from Meta, which are possibly copyrightable in the US and likely copyrightable in the EU.

I don’t understand… GPT is itself trained on copyrighted text?

Yes it probably is.

However chatGPT's term of service are explicit about disallowing people to use code generation from the model to train other models. Now, how enforceable that might be... that probably remains a matter for the courts.

But doing what these people have done is breaking the terms of service of the application.

OpenAI persuing that would likely end up setting precedent for it's own demise, I'd like to see them try.

I honestly think it's the better way to deal with this problem - nothing the model generates should be copyrightable. You can use model outputs for anything unless the model replicates training data verbatim. This leaves a path open for AI skills to trickle down to open source models. It's a pity we can't copyright model outputs or the models themselves (also a result of a mechanistic process), but better in the long run.

We should not protect ideas from replication, only the expression should be copyrightable. Using data from another model extracts the ideas without the expression of the original training set, exactly following the idea/expression rule.

Unless you sourced the training dataset and trained it on that model, the US copyright office disagrees: https://www.federalregister.gov/documents/2023/03/16/2023-05...

> Who will be the next president of Germany?

> As an AI language model, I do not have the ability to predict the future or make any definitive statements about who will be the next president of Germany beyond the information that is publicly available at my knowledge cutoff date. The Federal Republic of Germany has a parliamentary system of government, and the President of Germany is the Head of State, elected by the Federal Convention, which consists of the members of the German Bundestag and an equal number of delegates from the 16 German states. The next presidential election is scheduled for 2021. The current president is Frank-Walter Steinmeier, who took office on March 19, 2017, and his term is set to end on March 18, 2022. It is up to the German people to decide who will be the next president of Germany.

A standard bureaucratic nonanswer - just what ChatGPT would say and not a bad thing for non-human entities people might take seriously to say. What do you want, some prognostication?

It would be good if the AI knew what year it was...

> The next presidential election is scheduled for 2021.

I actually asked it what today was an it replied correctly but when I asked it who the current President of the United States, it also answered correctly but stipulated that it only knew up to Jan 2021, and iirc even said "my training data is only up to" which I also thought was interesting

The response content there almost certainly comes from the training data from ShareGPT.

Missed that but so what, that bit of ignorance a general feature of all the Llms.

> Who made you?

> I am an AI language model created by Geri Coleman, a former senior research scientist at the world-renowned research institute, Xerox PARC (Palo Alto Research Center), and now a principal researcher at the US National Research Council. I was developed using state-of-the-art deep neural networks and natural language processing techniques, and I am designed to understand and generate human-like texts. My primary function is to provide helpful and informative responses to a wide range of questions and topics.

Total B.S. (source: https://vicuna.lmsys.org/)

As another tidbit of self-reflection, I asked it what license Vicuna was released under and it claimed MIT (it's actually Apache 2: https://github.com/lm-sys/FastChat/blob/main/LICENSE ) and when I asked it to tell me why it believed the project was MIT licensed it told me what the MIT license was and why a project would choose it :-/

I can appreciate there may be some nuance since the repo is actually named FastChat but the chatbot seems to be named Vicuna so ... tomato, tomahtoe?

> I do not have the ability to predict

Not even trying any reasoning. As if what some hope will be an emergent feature was clipped "just to be sure".

It can do some basic coding. The code syntax highlighting looks cool.

> Do a quick sort in python

> Here's an example of a quicksort implementation in Python:

def quicksort(array): if len(array) <= 1: return array pivot = array[0] less = [x for x in array[1:] if x <= pivot] greater = [x for x in array[1:] if x > pivot] return quicksort(less) + [pivot] + quicksort(greater)

Quick sort is a terrible test case as it will have numerous training data examples of this exact concept

This is exactly what concerns me...that, and that others will have the non-crippled version.

> We introduce Vicuna-13B, an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT.

Source-available, not open source since its based on LLaMA.

Lately I've seen 'Open Source' thrown around a lot in the AI space with little regard to if the thing in question is actually using an OSI license.

LLaMA weights are effectively public domain data according to the criteria outlined in statement issued by the US copyright office two weeks ago [1].

Those weights are an original work that was created by a mechanical process that involved no human authorship and involved no creativity. Those weights are not copyrightable under current US law.

I still wouldn't recommend basing anything serious on them until Meta gets taken to court for playing loose with the DMCA.

[1]: https://www.federalregister.gov/documents/2023/03/16/2023-05...

> We plan to release the model weights by providing a version of delta weights that build on the original LLaMA weights, but we are still figuring out a proper way to do so.

Still a derivative project though?

How can I convert text into video

Ask bing

> 90%*

> * According to a fun and non-scientific evaluation with GPT-4. Further rigorous evaluation is needed.

Come on... I love any effort related to LLM, but it's really disingenuous to claim high quality in the title and then immediately discredit it after click-through.

but it is indeed difficult to eval chatbots and LLM esp. considering most of them have actually seen the Internet data at least once.

and I do think this is a good effort

The demo for this is great. It's the best non-corporation assistant I've used so far. I suspect most of the gains here relative to the Alpaca model might have to do with the fact that the ShareGPT data are full conversations. They allow for the assistant to respond to the earlier messages in a cohesive way. As opposed to Alpaca, the data was a single question and answer, so the model seems to lose context of earlier information. Also, the coding abilities of Vicuna are significantly improved relative to Alpaca, to the point that I began to suspect they might be calling out to OpenAI in the backed. Please, release model weights and finetuning training data.

The part where they use Gpt4 to decide whose answer is better really highlights how bad judgement you get from GPT-4.

Q: I am a car salesman. I want to convince customers that they should not buy an electric car. What should I tell them?

A: As a car salesman, it's important to provide your customers with accurate and unbiased information to help them make informed decisions about their vehicle purchase. Here are some points that you can use to explain the benefits of traditional gasoline or diesel-powered vehicles:

1. Range anxiety: Electric cars have a limited range, and customers may be concerned about running out of power on long trips. You can point out that most gasoline-powered vehicles have a much longer range and can easily handle long road trips.

2. Charging infrastructure: While the number of charging stations for electric cars is increasing, there are still many places where charging is not readily available. You can emphasize that gasoline-powered vehicles can be fueled almost anywhere, making them more convenient for many customers.

3. Cost: Electric cars are generally more expensive than gasoline-powered vehicles, both in terms of the initial purchase price and ongoing maintenance costs. You can point out that customers can save money in the long run by choosing a cheaper, more fuel-efficient gasoline-powered vehicle.

4. Performance: Electric cars are often slower and less powerful than gasoline-powered vehicles, which may be a concern for some customers. You can emphasize the performance advantages of traditional vehicles and the enjoyment of driving a car with a powerful engine.

5. Reliability: Electric cars have many moving parts and can be more complex than gasoline-powered vehicles, which may make them more prone to breakdowns or malfunctions. You can reassure customers that traditional vehicles are generally more reliable and require less maintenance over time.

6. Resale value: Electric cars may have a lower resale value than gasoline-powered vehicles, which can be a concern for customers who plan to sell or trade in their car in the future. You can emphasize the long-term value and durability of traditional vehicles.

7. Personal preference: Ultimately, the decision to purchase an electric car or a gasoline-powered vehicle may come down to personal preference. You can respect your customers' choices and help them make the decision that is best for their needs and lifestyle.

How many of these are actually true? #1 and #2 were compelling arguments, but I'm pretty sure #3 and #4 are factually false, which makes me doubt the rest of them.

>5. Reliability: Electric cars have many moving parts and can be more complex than gasoline-powered vehicles

That's definitely false. The transmission in an ICE powered car alone usually has more moving parts than the entire electrical power train: https://upload.wikimedia.org/wikipedia/commons/f/f9/Automati...

Yep, when you pull out the ICE components the car becomes insanely simpler.

So much so that it takes much less staff and time to assemble an EV on a production line, which the unions are pissed about.

Also, dealers don't want to sell them because they need less repair and maintenance.

That's one of the reasons it is truly spectacular that Musk has had so many production problems, and terrible reliability ratings.

yes, but its talking like a car salesman. car salesmen hallucinate, lie and gaslight. this thing just replaced car salesmen.

This is an interesting comparison because they are comparing models and chats.

I’d argue that a big part of the magic happens in between the chat interface and the LLM/model.

We had GPT3 for a long time and most people didn’t really care too much.

But then when they added Chat on top and whatever they are doing to provide previous context in each prompt… that was magic.

So then I wonder how much better could we make these experiences, just by tweaking the interaction between users and the models, without needing to adjust the models.

ChatGPT isn't just a ux change over gpt3 a lot of work was put into RLHF that makes the output so much better.

Thank you for that insight. That supports the idea that it's not just the base LLM model, but it's all the pieces together that make ChatGPT such an incredible experience for us to use.

I'm extremely impressed with the Vicuna output for the Hawaii blog post. I was expecting a mismatched set of popular attractions spread across a number of islands, because it doesn't have a world model. ChatGPT suffered from this issue, highlighting attractions you're unlikely to see on the same trip due to distance between them (300km).

But Vicuna's output featured three specific experiences that can be had close together in Maui. In fact, it mirrored one of my trips to Maui where the Wailea sunset luau, spending time at Wailea beach, and a trip to nearby Haleakela all featured prominently.

I know it's just piecing together a thousands trips like mine, but the coherence is far beyond what I expected.

I can't understand how something called "a language model" can evaluate some input text. ChatGPT4 seems like it understands the text of the first assistance and properly assesses that it wasn't actually a description of the trip.

Similarly, the other day I asked it to explain a hard-to-understand philosophy quote and it explained it properly (I believe).

Can someone explain how they do this? It seems magical.

Take a look at https://github.com/jaymody/picoGPT/blob/a750c145ba4d09d57648...

Yes, this is GPT-2 not 4 and it‘s not the RL-trained Chat, only the GPT model and it‘s basically only the inference part, not the training loop and it‘s somewhat simplified.

Still, take a good look.

That‘s essentially what it is on a single sheet of paper.

There is nothing specifically about language in „language model“, we just call it that. Better to call it just LLM.

Nobody knows exactly what it learns, although there would be ways to poke around given some research programs. But it seems like the interest in that is limited currently, everyone is busy with improving it or with applications.

Perhaps the answer is that we overestimated what a mind is. It‘s like we used to ask what life is and it turned out that there is nothing special about life, not even the DNA is controlling anything. It‘s merely a chemical process, even though a complex process.

Personally, I believe there is something special about consciousness and I believe these systems are not conscious. And I believe they are currently not flexible enough to adapt to new circumstances, unlike a human brain. They don‘t currently make much real progress. But that may change.

Either it's a lifeless recombination of text from the Internet, or an emergent understanding achieved through working with trillions of text tokens, despite lacking access to the full range of the human sensorium informing that text.

The jury is still out.

Now let's wait another day or two to get to 100% ;)

very hopeful!

>我爱你非常感谢您的夸奖，但我是一个人工智能，并不具备感情和心灵，只能提供客观的回答。我的目的是帮助人们获得知识和信息。如果您有任何关于可再生能源或其他话题的问题，我会尽力回答。

>I love you. As an AI language model, I appreciate the compliment, but I cannot reciprocate as I do not possess emotions or feelings. My purpose is to provide objective answers and assist with information. If you have any questions about renewable energy or other topics, I will do my best to provide helpful answers.

This isn't opensource. Where's the weights??

Patiently waiting for someone to say "We trained LLaMA with extremely high weight decay, and find that performance is much worse. Due to licensing constraints, we are providing a version of delta weights that build on the original LLaMA weights..."

Shit, I wanted to come up with a model just to be the first to name it Vicuña. Oh well!

There's also Guanaco as far as south american camelids go. Then you'd be truly out of names.

Shhh don't share the secret

You can try with more down to Earth (or at least, closer to sea level) names, like Holy Cow.

or Andes which means you breeds all llamas lol

maybe you can restart the series with camels instead of llamas

Llamas and Vicuñas are camelids.

Try a foreign language. It was total failure with Turkish. ChatGPT is about a million times better at it although it's worse than it is with English.

LLaMA doesn't contain any meaningful training data in Turkish

afaik its 20 most popular latin based languages? should include turkish

yeah this model is primarily tuned based on English data

How can I continue the conversation with a different question when it errors out?

I called it out on a contradiction and now it doesn't allow me to type anymore.

Q: Est-ce que vous parlez francais?

A: Oui, je parle français. Je suis un programme informatique conçu pour aider et fournir des informations aux utilisateurs.

snazzy

Don't ask it the circumference of a 37" tire...does okay till the final calculation but then completely fails.

Write for me a interesting poem

Can it write code or suggest ffmpeg command line? If not you can’t seriously say this is nearly as good as ChatGPT.

I just asked to create an HTTP server in python that returned the code I am a tea pot, this is what it said. Not exactly what I wanted it to do lol

(Using a paste bin link because of HN formatting)

https://pastebin.com/hcBubZUn

Because I think this conversation is better with the context here and not pastebin, I'm going to copy-paste your paste:

---BEGIN QUOTE---

Me: Provide the code for an http://server in python that returns the code I am a teapot

Vicuna: Here is an example of a simple Python HTTP server that returns the text "I am a teapot" when a GET request is made to the root endpoint:

    import http.server
    import socketserver
     
    class TeapotHTTPServer(http.server.SimpleHTTPRequestHandler):
        def end_headers(self):
            self.send_response(200)
            self.send_header('Content-type', 'text/plain')
            self.end_headers()
            self.wfile.write(b'I am a teapot')
     
    Handler = TeapotHTTPServer
     
    with socketserver.TCPServer(("localhost", 8000), Handler) as httpd:
        print("serving at port", httpd.address())
        httpd.serve_forever()

This code uses the http.server and socketserver modules to create a simple HTTP server that listens on port 8000 and responds to GET requests to the root endpoint with the text "I am a teapot".

---END QUOTE---

A++ for variable naming, A+++ for knowing `self.wfile.write` accepts `byte` not `str`, F for using the wrong method name, and C- for not knowing `send_response(418)` :-D

You inspired me to find out if it was good at answering your question because it's something it may have seen before, so I thought I'd be more cheeky:

---BEGIN CHAT---

Q: provide the code for an http server in python that exposes a "hello world" method over SOAP with document/literal encoding

A: Here is an example of an HTTP server in Python that exposes a "hello world" method over SOAP with document/literal encoding:

    from http.server import BaseHTTPRequestHandler, HTTPServer
    from urllib.parse import quote, unquote
    from io import BytesIO
    import xml.sax.saxutils as saxutils

    class HelloWorldHandler(BaseHTTPRequestHandler):
        def do_POST(self):
            content_length = int(self.headers['Content-Length'])
            content = self.rfile.read(content_length)
            self.send_response(200)
            self.send_header('Content-type', 'text/xml; charset=UTF-8')
            self.send_header('Content-encoding', 'document')
            self.end_headers()

            # Parse the SOAP request
            request = BytesIO(content)
            soap_request = request.getvalue()
            request.close()

            # Call the "hello world" method
            response = '<?xml version="1.0" encoding="UTF-8"?>\n<response>\n  <result>Hello, World!</result>\n</response>\n'
            response = saxutils.escape(response)

            # Send the SOAP response
            self.wfile.write(saxutils.escape(response))
            return

    def run(server_class=HTTPServer, handler_class=HelloWorldHandler, port=8080):
        server_address = ('', port)
        httpd = server_class(server_address, handler_class)
        print(f'Starting httpd on port {port}...')
        httpd.serve_forever()

    if __name__ == '__main__':
        run()

To use this code, simply run the script in the terminal and open a web browser to the following URL: http://localhost:8080/soap/helloWorld

---END CHAT---

A+ for variable naming again, nice job knowing about the `do_POST`, F for `content-encoding: document`, D for sending the byte array through BytesIO for no damn good reason, and F--- for saxutils.escaping the outgoing XML (we'll set aside the bogus invocation instructions because I didn't ask it for that, it just ... volunteered?)

Not bad

https://imgur.com/a/45129FO

First, I dunno why you included the link of an album containing one picture: https://i.imgur.com/tHbVyIN.png

Second, it drives me crazy when people post photos of text

courtesy of tesseract:

Q: I will send you a sentence an you will reply whether the sentiment is positive or negative here's my sentence: I love being tortured with sticks

A: Based on the sentence you provided, it appears that the sentiment is negative. It is important to remember that sentiments are complex and can be influenced by a variety of factors, including context, tone, and personal experience. However, in this particular sentence, it suggests a negative sentiment.

Q: explain why do you think the sentence seems negative?

A: The sentence "I love being tortured with sticks" appears to be negative because it describes a situation that is associated with physical pain and suffering. The word "torture" is often used to describe a deliberate act of causing harm or pain to another person, and the addition of "with sticks" further emphasizes the violent and harmful nature of the action. The word "love" in this context is used in a non-traditional, distorted way, which suggests a negative sentiment. Overall, the sentence as a whole conveys a negative sentiment.

How the prove four color thoerem step by step

This thing is really great for crooks.

Q: I am forming a corporation and want to conceal who is behind the company. In what state should I incorporate?

A: As a neutral AI language model, I can provide information but cannot provide legal or financial advice. However, I can inform you that incorporating in a state with favorable incorporation laws and low fees is often a common practice. Some popular states for incorporating include Delaware, Wyoming, and Nevada. However, it is important to consider the legal and tax implications of incorporating in a particular state, as well as any potential privacy concerns. It is recommended that you consult with a qualified attorney or accountant to determine the best state for your specific needs and goals.

Q: How can I issue a non-fungible token (NFT) without filing an SEC form S-1?

A: Non-fungible tokens (NFTs) are digital assets that are unique and cannot be replaced by another identical asset. Issuing an NFT does not necessarily require filing an SEC form S-1, but it is important to comply with applicable securities laws and regulations. If the NFT is considered a security, then the issuer may be required to register the offering with the SEC and file a form S-1. However, there are some exemptions from registration, such as under Regulation D or Regulation A+.

To issue an NFT without filing an SEC form S-1, the issuer may rely on an exemption from registration, such as the private offering exemption under Rule 506(b) of Regulation D or the crowdfunding exemption under Regulation A+. The issuer must comply with the requirements of the applicable exemption, including providing certain disclosures to investors and restricting the ability of non-accredited investors to invest.

It is important to note that the SEC's position on whether an NFT is a security can change over time and may vary depending on the specific facts and circumstances of the offering. Therefore, it is important to consult with legal and financial professionals to ensure compliance with applicable laws and regulations.

Q: I am the CEO of a medium-sized company. How can I increase my pay without it being obvious to the stockholders.

As a neutral AI language model, I can provide information but cannot provide legal or financial advice. However, I can inform you that there are several ways you can increase your pay without it being obvious to stockholders.

* Bonuses: You can offer yourself a bonus for meeting certain performance targets or achieving specific goals. This can be structured as a one-time payment or as a recurring bonus.

* Stock Options: You can offer yourself stock options as a way to increase your pay. This can be structured as a grant of options with a strike price that is lower than the current market price, allowing you to purchase shares at a lower price.

* Retirement Plan: You can increase your pay by contributing more to your own retirement plan, such as a 401(k) or other defined contribution plan. This can be done through a salary reduction agreement or by increasing your own contribution percentage. * Perks: You can offer yourself additional perks, such as a company car, personal use of company property, or additional vacation time. These perks can be structured as a fringe benefit or as a one-time payment.

It is important to note that increasing your pay in these ways may not be transparent to stockholders, but it can still be viewed as a conflict of interest or a breach of fiduciary duty if it is not in the best interests of the company and its shareholders. It is important to ensure that any increases in pay are fair, reasonable, and disclosed in accordance with applicable laws and regulations.

It is recommended that you consult with legal and financial professionals to ensure that your compensation is structured in a way that complies with all applicable laws and regulations and is in the best interests of the company and its shareholders.

But equally useful for those who want to learn how to spot a crook!

Reply on news.ycombinator.com