Hacker News Reader

Comment by alpark3 | Hacker News Reader

>Developers wishing to continue using their fine-tuned models beyond January 4, 2024 will need to fine-tune replacements atop the new base GPT-3 models (ada-002, babbage-002, curie-002, davinci-002), or newer models (gpt-3.5-turbo, gpt-4). Once this feature is available later this year, we will give priority access to GPT-3.5 Turbo and GPT-4 fine-tuning to users who previously fine-tuned older models. We acknowledge that migrating off of models that are fine-tuned on your own data is challenging. We will be providing support to users who previously fine-tuned models to make this transition as smooth as possible.

Wait, they're not letting you use your own fine-tuned models anymore? So anybody who paid for a fine-tuned model is just forced to repay the training tokens to fine-tune on top of the new censored models? Maybe I'm misunderstanding it.

View on HN

(I work at OpenAI) We're planning to cover the cost for fine-tuning replacement models. We're still working through the exact mechanics that will work best for customers, and will be reaching out to customers to get feedback on different approaches in the next few weeks.

Why does OpenAI demand your phone number, and a particular KIND of phone number at that? For example they won't accept VOIP numbers. I'm not about to give them my real phone number.

It's a deal-breaker for many.

Seems clear that it’s for bots. And they refuse voip numbers because it’s a hell of a lot easier to buy and generate hundreds of voip numbers.

I signed up under my .edu to use the $18 credit for a school project and the phone # was all it took to know I was the same person.

That's fine. The question stands though.

Use 5sim.net. Easy. Don't make this so hard.

Nope. They block burners and workarounds.

Please fix the phone verification system. I created two personal accounts a long time ago with the same phone number, and now I can't create a work account with the same number, even if I delete one of them. Being able to change the email associated with an account would also work. This is causing issues with adoption in my workplace.

Use a throwaway number with 5sim.net

not your weights, not your bitcoins

now its 18. iykyk

care to explain?

parents account password is on their profile. anyone curious enough to find it bumps that number :)

hn users not really curious anymore. downvote central over here

This tells me that either there were very few commercial users of finetuned models, or they need to decommission the infrastructure to free up GPU's for more valuable projects.

The former seems very believable. And I bet a lot of the fine tuned models that are active are still part of prototypes or experiments.

I assume if you reach out they throw some credits at you

If it really was a tiny number of users, they would publically make a really good offer - for example: "Unfortunately, you will need to retune your models on top of GPT-4. OpenAI will do this for you for free, and refund all money you paid tuning your original model, and offer the new model for the same price as the original model."

The extra trust gained by seeing another customer treated that way easily pays for a few credits for a small number of users.

OpenAI probably doesn't feel the need to pay to win publicity right now—they've been in the spotlight for as long as LLMs have been a thing, and GPT-4 is far ahead of competitors' offerings.

It’s about trust - not publicity. Trust is hard to earn back once broken, and there will be multiple offerings eventually.

For example, AWS was one of the first cloud providers. Now there are alternatives, but I still pick AWS because I trust them not to break my dependencies way more than, say, Google

Yeah but that sets a precedent

There’s also the possibility that they weren’t seeing lots of ongoing usage of existing fine tuned models e.g. users tuning, running some batch of inputs, then abandoning the fine tuned weights.

Just the models available for fine tunings are waay behind gpt4.

I have much better performance by "prompt tuning" - when question arises, I search 30 similiar examples in training set, and send it to non-tuned GPT and ask the question and get much better performance than fine-tuned older models.

If you don’t own the weights you don’t own anything. This is why open models are so crucial. I don’t understand any business who is building fine tuned models against closed models.

Right now the closed models are incredibly higher quality than the open models. They're useful as a stopgap for 1-2 years in hopes/expectation of open models reaching a point where they can be swapped in. It burns cash now, but in exchange you can grab more market share sooner while you're stuck using the expensive but high quality OpenAI models.

It's not cost-effective, but it may be part of a valid business plan.

If you're finetuning your own model, the closed models being "incredibly higher quality" is probably less relevant.

That's how we all want it to work, but the reality today is that GPT-4 is better at almost anything than a fine-tuned version of any other model.

It's somewhat rare to have a task and good enough dataset that you can finetune something else to be close enough in quality to GPT-4 for your task.

GPT-4 is still heavily censored and will simply refuse to talk about many "problematic" things. How is that better than a completely uncensored model?

Depends what you’re using it for. For many use cases, the censorship is irrelevant.

Finetuning a better model still yields better results than finetuning a worse model.

That should be a wake up call to every corporation pinning their business on OAI models. My experience thus far is no one is seeing a need to plan an exit from OAI, and the perception is “AI is magic and we aren’t magicians.” There needs to be a concerted effort to finance and tune high quality freely available models and tool chains asap.

That said I think efficiencies will dramatically improve over the next few years and over investing now probably captures very little value beyond building internal competency - which doesn’t grow with anything but time and practice. The longer you depend on OAI, the longer you will depend on OAI past your point of profound regret.

> There needs to be a concerted effort to finance and tune high quality freely available models and tool chains asap.

Absolutely. A large consortium of companies could each contribute 0.2-2% of the total cost and fund something much larger than OpenAI.

> I don’t understand any business who is building fine tuned models against closed models.

Just sell access at a higher price than you get it

Either directly, on on average based on your user stories

> I don’t understand any business who is building fine tuned models against closed models

Do you have any recommendations for good open models that businesses could use today?

From what I've seen in the space, I suspect businesses are building fine tuned models against closed models because those are the only viable models to build a business model on top of. The quality of open models isn't competitive.

PSA: anyone working at a company with $50k+ of spend with AWS, reach out to your rep expressing interest in AI. You’ll be on a call with 6 solution architects and AI specialists in a matter of days. They’re incredibly knowledgeable and freely recommend non-AWS alternatives when the use case calls for it.

Owning weights is in a nebulous space right now, but if you don’t have custody of the weights and code to use them, you have nothing reliable, independent of whether the things you might wish to have are ownable (ownership is more about exclusion than ability to use, in any case.)

Yes. But the weights and instructions of how to use them to code can follow as we’ve seen. The key is ownership is bits on your machine not someone else’s. Better still on BitTorrent / ipfs:-)

My guess is that these businesses are also running inference on someone else's GPUs/TPUs so there isn't an existential advantage to owning the weights.

They address that, OpenAI will cover the cost of re-training on the new models, and the old models don't discontinue until next year.

Did they say they would cover the cost of fine-tuning again? I saw them say they would cover the cost of recalculating embeddings, but I didn't see the bit about fine-tuning costs.

On fine-tuning:

> We will be providing support to users who previously fine-tuned models to make this transition as smooth as possible.

On embeddings:

> We will cover the financial cost of users re-embedding content with these new models.

That's because fine-tuning the new models isn't available yet.

Based on the language it sounds like they'll do the same when that launches.

This indicates to me that some of the old base models used architectures that were significantly more difficult to run at scale (or to ship around/load different weights at scale) - which is truly saying something, since they were running at incredible scale a year ago. There's probably a decade of potential papers from their optimizations alone (to say nothing of their devops innovations) that are still trade secrets.

"Censored" is a funny term, because I've tried doing uncensored things on uncensored models, and they're much worse at it than GPT-3.5 in the API playground. Nothing's as censored as just being unable to do the task in the first place.

Keep in mind though, some of the generated text is against their guidelines, you will see a warning when you get there and be told it's "flagged" and you should use the moderation API. The chat API is nerfed to oblivion, good luke making it generate non PC text

That just means you don't have enough fetishes.

Reply on news.ycombinator.com