Hacker News Reader

Comment by xwdv | Hacker News Reader

Given the high prices of OpenAI offerings it seems it’s better to pirate an AI model before resorting to paying for anything.

View on HN

The world is becoming more cyberpunk everyday, people making back alley deals for data or models.

These weights are shit man, they have been quanted w/o being retrained against the original. I already have this torrent, I want uncut originals. And no water marks this time, the last model wouldn't shutup about investing in tulips.

Shh, I got some unnerfed midjourney v8 for your nsfw needs, my friend...

I will only pay half unless you supply the training data as well.

ChatGPT API is surprisingly cheap, but GPT-4 is many times more expensive to the point where I can't see it being worth it most of the time.

I'm not convinced that the GPT-4 API actually works? It's been giving me very different answers than the chat interface. For example, the chat interface says it's GPT-4 if you ask it, but the API says it's GPT-3 (and bills as GPT-4).

If you're satisfied with the results who cares what it thinks it is

I'm not. It's worse than the chat version.

Why would the LLM model know what version it actually is? It was trained on data from before that model version even existed.

- What version model are you?

- I am based on the GPT-4 architecture, which is an iteration of OpenAI's Generative Pre-trained Transformer models. My training data includes information up to September 2021.

- How do you know your version?

- As an AI language model, I am programmed with information about my own architecture and the version I am based on. My creators at OpenAI provided me with this knowledge, which helps me to give context to users about my capabilities and limitations. In this case, I am based on the GPT-4 architecture, an iteration of OpenAI's Generative Pre-trained Transformer models.

Likely making this up.

Why did it not make up GPT-5 or something else, though?

Try asking it several times with slightly different phrasings.

I don't have all the answers!

Has the ChatGPT-4 “system” prompt been revealed?

Hmm, you think that's what makes a difference? It might be, though I've gotten different (what I find worse) results on other questions as well.

This is a great (pre-GPT-4) post that goes into the system prompt and its effects: https://minimaxir.com/2023/03/new-chatgpt-overlord/

Are you using any system prompt in your API calls? Note that it does have an explicit domain separation from the user and assistant roles.

I suspect it plays a greater role in GPT-4 as well, i.e. was trained on a wider variety of system prompts to guide the output of the model.

I wasn't, no. You're right, that might make a difference. Thanks for the article, it's very useful, I've been curious about the system prompt!

How much does it cost to privately fine-tune and run Llama?

It’s USD 600 for fine-tuning. Maybe USD 4-5k for a computer that can run it.

ChatGPT pro is $20/month. 5k would be 250 months (10+ years) of paid access.

Not sure pirating it now adds up.

Correct me if I'm wrong, but that's per-user, unless you all just share an account, or you build out a bespoke API integration. Which is to say if you have 25 developers, you'd spend $5k in 1 month.

The reason to pirate it would to be able to fine-tune the model on your private internal source code repository, assuming you already have an existing large body of work you want to train it and offer SelfHostedCoPilot on your bespoke internal-only DSL that ChatGPT and Copilot has no way of having seen and would undoubtedly hallucinate about by a considerable margin.

Most people doesn’t fine tune the models (llama or openAI). A MacBook M1 can run those model ($1000) and in many cases the user already have it. You also need a computer to access openAI, the comparison boils down to $20/m vs $0. At this point in time, LLM are a curiosity for most people

Isn't openAI only charging something like a 20th of a penny per interaction right now? Definitely not the kind of thing you want to incorporate into a widespread free app just yet, but it seems pretty affordable for a lot of use cases

That's the price per "token". A token is a word or part of a word - rule of thumb is four tokens per three words.

> Isn't openAI only charging something like a 20th of a penny per interaction right now?

They don't charge per interaction, but per token. The chat models range from a fifth of a cent per 1000 tokens to 12 cents per thousand tokens (depending on whether it's gpt-3.5, or the 8k limit gpt-4, or the 32k limit gpt-4, and, for gpt-4 models, also prompt v. response tokens.)

Reply on news.ycombinator.com