OpenAI delays launch of custom GPT store until early 2024 | Hacker News Reader

OpenAI delays launch of custom GPT store until early 2024

axios.com

106 pointscloudking65 comments

The release of plugins was underwhelming for a lot of users, I wonder if they're worried about the same thing happening again. I had fun making a plugin when they were first announced but the developer experience was clunky and frustrating, and there was a lot of confusion around how the plugin store actually worked.

I haven't made the time to convert my plugin into a GPT but I'm planning to, hopefully it'll be a bit of a smoother experience this go around.

I suspect they're a bit like Google... wonderful tech but terrible product company.

Everything outside of the quality of their models is suspect:

- GPTs are super quirky w/ weird limitations... try processing an image file larger than an emoji from base 64 text and watch it render the base64 string in a code window and give up after about 1k tokens. The vast majority of the time it doesn't even try and hallucinates processing what it thinks a file related to the context of the GPT would do.

- The assistants API is a black box of inefficiency and out of control costs due to a propensity to barf huge amounts of context from problematic RAG, etc. Yet it's not obvious in real time because there is no token consumption information like the normal chat API. It's almost like they rolled it out like this to destroy developer trust on the spot by doing a 'gotcha' to anyone naive to roll this out to prod without sufficient testing.

I'm honestly unsure how GPTs will in fact be monetized. Are purchasers required to have a Plus account w/ sellers getting a one time sale? Will sellers have to pay extra usage fees, or a shared subscription type thing split between them and OpenAI?

I think they should just give up on this area of the business. Dev day produced some anxiety by encroaching on the business models of a large segment of newly funded AI startups... then showing the efforts to be half-baked while encouraging more lock-in just is going to erode trust and uptake on API usage in general, possibly having people be relieved when Google Gemini and similar efforts are unveiled in the coming months.

It really seems like it's there to take any low hanging fruit away from the startup market and continue riding the hype train as long as possible while paradoxically also bringing to light just how useless GPTs are for most serious automation tasks.

> wonderful tech but terrible product company

i couldn't have said it better, something i've been feeling for a long time. they have all the engineering expertise, why not the same for product?

it's even more surprising given Altman's ycom experience, it's not like this is a classic engineer-run corp

On top of that the CTO has a product, not engineering background. It doesn't make a whole lot of sense.

but the president was the CTO of Stripe

Who I understand has no reports and mostly codes, but yes he is tight with Sam and undoubtedly has a big influence over strategy.

Might be underwhelming because the rollout of allowing authors was extremely limited? That’s how I felt.

I pay for ChatGPT and never got off the waitlist when first announced. Killed all hype for me.

The UX itself isn't good anyway.

I tried to use this is to create a travel itinerary, TBH, it is the wrong interface for this task, it is slow and it is full of useless information. Ain't good.

The most useful one is probably the PDF reader plugin but now that is part of ChatGPT itself.

Yes and having to manually toggle the plugins and hope that your requests end up in the right one wasn’t great either. Especially with the 1 action per response limit. The new GPTs look and feel much more, for lack of a better word, natural. I‘ve had an interaction in the main GPT where it first looked up something on the internet and then followed up with generating an image and writing a suitable text. I found it pretty magical.

As far as UX, and a product, chatGPT is terrible!

Astounding and blows my mind for a 80+b dollar company. The model and tech (and team) is that value of course, but how can't they at the very most basic level, clean up the 'sidebar'. Prompts search and GPT labelling. Even when they had the plugins, most ppl didn't know you had to click on the GPT4 button to revial, because why would they...nothing indicated to (like an arrow). Total amateur hr in the UX and DX dept and they treat they're paid customers like trash.

If this is how Sam Altman thought product leaders at YC to build product I'm shocked. Cause OpenAI is way behind where they should be will literally a million dollars thrown at improving this things in a cpl weeks...and they've had a yr

And just imagine the outrageously overwrought, self-flagellating, beard-stroking, sophomoric, herculean UX interview process they torture designers with just to make...that.

I think they will end up buying Cohere, Cohere has a great product team and is an obvious fit.

This says a lot more about the state of the frontend ecosystem than it does about the OpenAI dev team. The complexity has gotten absurdly out of control.

It's really quite ironic that on one hand, we've got a company on the cusp of developing a superintelligence, and a bunch of open source models lagging not too far behind. But on the other hand, you've gotta install it all with Python - and good luck, because nobody in the ecosystem is pinning dependencies and you're gonna have to re-install the thing every time you run it.

And then on the frontend, you've got the team responsible for creating the interface to the super intelligence. They've actually done a decent job thus far, but it's slow development and disconnected from more ambitious scaling ideas.

Are we calling language models "super intelligences" now?

> This says a lot more about the state of the frontend ecosystem than it does about the OpenAI dev team. The complexity has gotten absurdly out of control.

That’s a peculiar way to spin it.. not sure how doesn’t that justify poor UX though (what does this even have to do with python)

> but it's slow development and disconnected

It’s as slow and disconnected as they want it to be

Still dont understand why anyone would "pay" for this. The default model will probably be better in pretty much all cases. I guess if someone randomly has some sort of esoteric knowledge? Seems very limited.

Is a custom GPT just a boatload of prompts injected by default?

Not sure how much you've used custom GPTs. They are currently flawed and buggy but I think they hold promise. They combine 3 things:

1. A custom initial prompt (yeah - you can paste this in every time and keep a personal database of special purpose prompts - but it rather kills the convenience and utility)

2. A JSON description of one or more APIs that can be used (not just for retrieveal of information - but potentially to take actions. This aspect is under-explored IMHO)

3. Uploaded documents which (I presume) are encoded into a vector database. This part is currently the most flaky. I was hoping this was a way to get round context-window limits. It has worked very badly so far and I'm not sure I understand why.

If these things start working effectively in combination then there's potentially a "the whole is greater than the sum of it's parts" going on here. Each of these things is fairly useful on it's own but if you combine them and have them on tap - and someone else has put the hours in to making them useful and well-tuned - then it is potentially transformative.

Yes - GPT on it's own is pretty good at a lot of things, but you spend a long time working around it's flaws and blind-spots.

Custom GPTs don't cure all the warts but they are still potentially a force-multiplier.

EDIT - I'm also currently underwhelmed how well the other "special powers" work together. Image generation can't be fed into Image Understandin. Python is hamstrung in various ways. And GPT has no real understanding of how to prompt Dall-E. In fact it's worse at prompting than I am.

These flaws are potentially fixable. I hope.

I guess. I assume the OpenAI vision, in the short term, is for there to be a beefy LLM on top that can call the correct sub LLMs that have more bespoke knowledge about random things, and sort of orchestrate an answer based on that.

Not sure why OpenAI feels the need to pay people for this though, I guess to incentivize use?

They all use a beefy LLM though.

3. working seamlessly would be a major thing. 1. and 2.? I used both, together, for months, via TypingMind.com (alternative frontend UI, bring-your-own-API-key).

This is not to advertise the service - even though it does deserve it, at the very least for having a perpetual license instead of subscription bullshit - just to say that a) OpenAI is lagging way behind the obvious in everything surrounding the models themselves, and b) the combination of 1. and 2. is convenient, but not ground-breaking. At least, not unless you hook a powerful enough "actuator" via 2. This, I agree, seems underexplored.

(Or underreported? It's an obvious lever to go after, a lot of people have said they're going after it, it's reasonable to expect some spectacular results from this, and yet... nothing. Crickets. Is everyone just staying silent and trying to build a bullshit startup to monetize whatever cool thing they made?)

ArxivXplorer works fairly well all considered... Bypasses the PDF and exposes math in a better format to the LLM.

They work OK if you don't set your expectations too high.

You can use Google as a replacement to find GPTs. For example:

"site:chat.openai.com/g/ <healthcare>"

I tried some amusing ones:

site:chat.openai.com/g/ alien

site:chat.openai.com/g/ uap

site:chat.openai.com/g/ agi

Found: An AI specializing in alien threat assessment with visuals and user grading. B

A "scoop"?! Each and every creator of a GPT has received an email with this announcement from OpenAI.

can you forward yours? I didn't get one for whatever reason. devin at techcrunch dot com

> Dear GPT Builder,

> It’s been less than a month since we announced GPTs and we are blown away by the useful and fun GPTs that you and the builder community have created.

> We are continuing to make improvements to GPTs based on your feedback. To improve Actions we updated the configuration interface, enabled one click testing, added debug messages in preview, and now allow multiple domains. There have also been questions around uploaded files. Uploaded files are downloadable when using Code Interpreter so we’ve made this feature default off and added messaging to better explain this. If you have additional feedback, we’d love to hear from you here.

> In terms of what’s next, we are now planning to launch the GPT Store early next year. While we had expected to release it this month, a few unexpected things have been keeping us busy! In the meantime, we will have some other great updates to ChatGPT soon. Thank you for investing time to build a GPT.

> - ChatGPT Team

appreciated, but do you mind forwarding the email itself? it's not that I don't trust you, just helpful to have. I won't use your email for anything.

edit: thanks to all who sent.

I don't mean to be harsh, but is this journalism now?

When you write an article and say something has been corroborated by your "sources", does that mean "forwarded to you by anonymous HN posters"?

You don't have an IRL relationship with a source who has created a GPT? You can't throw a stone in the valley without hitting somebody who has.

That has always been journalism. “Sources” encompasses all sources from whistleblowers to employee leaks to random overheard conversations on the internet or AFK.

I think the crux of the request was that they didn't get the email that "each and every creator of a GPT" was apparently supposed to get.

I've created almost a dozen GPTs that I and some others use almost every day, and I didn't get the email either. I think it's reasonable to ask for someone (even a stranger) to forward you a missing email.

Edit: Asked a friend and he didn't get any email either. ¯\_(ツ)_/¯

Thanks, I received the same email.

Can't wait for this new feudal lord to be replaced by open source.

You prefer an open source lord?

I... what? Yes?

Is anyone actively using custom GPTs? I've really tried to find a compelling use case but every thing I can think of is much better off as a simple system prompt you just copy and prepend (since it's easier to modify it, and it becomes more transparent).

I use it to avoid exactly that. Having to store, copy and paste various prompts over and over. I create one for each "domain" I operate with.

You can upload files. I've made one to act as a coding assistant to a little-known language that way. It seems to work OK. It makes syntax errors, but I'm not sure if that's a limitation of the Custom GPTs paradigm, or if the docs I gave it aren't any good. I know there are other services that do this kind of thing, but why bother finding one?

Can you say more about the information you uploaded for the language to be useful – for example, the language specification, or a "getting started with $LANG" manual? I'm tempted to do a GPT for the classic Macintosh toolkit, but since I'll need to do a lot of OCR I'd prefer to be intentional about what I spend my effort on.

I have so far found them “useful” mostly for having a collection of system prompts, and for sharing silly prompts with friends. It’s basically a way to attach a prompt to a URL.

For instance I have a “Beastie Bot” that plays a game where it categorizes things as “Beasts” or “Beasties.”

I have 2 custom GPTs, 1 of which I use every day called Snarky IT Wizard. I uploaded the PDFs for my particular router firmware version so I can ask questions, I've specifically configured it to be sarcastic and insert humor into both responses and the code it generates. Honestly, using it makes my day every time.

The other I was tailoring as a dungeon master for 5th edition, I've uploaded PDFs of all the source materials I can purchase.

My only gripe with the system is that as of late the whole openai platform has been experienced issues constantly and I get errors when trying to customize or even ask it questions from time to time.

I created one for my HOA. I uploaded bylaws, rules, assignments and made instructions for using them for policy. Eg. "What parking spaces are assigned to unit 123?" So far, I'm impressed and it's useful. BUT $20/mon is a deal killer and I don't know if the GPT Store will make it more affordable for our community.

My first thought when hearing about an "app store" or whatever this is: hopefully it's not inundated with garbage apps and marketing gimmicks.

I also think there's a risk of GPTs becoming so opaque as to not clearly understand any of their value

Version Control missing from GPTs is absurd, holy smokes, please enable GPT versioning

As is, actions in custom GPTs are almost useless right now, since you need to approve each and every request. Hopefully they fix that.

There's a form in the GPT where you can mark each action as "Ask" or... actually only "Ask", not deny or allow. On a single request I got a button that said "Always Allow" which I clicked... but only once and it did not do as advertised.

Anyway, clearly it's coming.

I’ve had the opposite problem, where it will initiate a request too early in a sequence defined in the instructions.

the customized GPTs are honestly a let down

you can't do anything fun with them, and that's not because it isn't possible but because openai is afraid to step outside of the safety zone. the response style really doesn't change much, default behavior leaks into everything so easily

what were they expecting?

Honestly, I am excited for people to re-implement the API for this using open models because of this. Maybe let the API stabilize a little, and i'm sure some implementations will pop-up.

What's the state of open models, though? I know that OpenAI's secret sauce is paying $$$ for access to private corpora, like the AP's archive, and I wonder how much the different levels of access to training data are reflected in any divide between open and commercial models, especially in different use cases like fact retrieval and synthesis.

(I've gotten too reliant on ChatGPT to satisfy my curiosity re: random questions – for example, "what are some French novels that depict post World War II Paris?" Honestly, if there's an open model that's beyond some "good enough" threshold – I don't know how to judge this – I'd download it on my computer ASAP.)

Heh, so it totally depends on the use case.

I use GPT4 constantly to chat through issues I am working on and get different perspectives. I cannot do that with local models.

On the other hand, I have been processing a ton of text transcripts with a fine tuned llama2 13b model i've been working on, and for the tasks I have fine-tuned on, my local model is producing better results than GPT4, often taking a task that I had to do in multiple steps with GPT4, and being able to complete it in a single shot.

I can run my local model through vLLM on my workstation at around the same tokens/sec as I can spend maxing out my API limits with GPT3.5-turbo (~$20/hr) while running on 2x 3090's. I'm hitting the vLLM (OpenAI clone) chat/completions endpoint. My model implements the HF chat_templates feature, and I worked on adding support for that to vLLM: https://github.com/vllm-project/vllm/pull/1756 (llama.cpp is talking about adding support for it too) so I could easily swap out my model in my data pipeline in place of GPT3.5/GPT4, and I wouldn't have to keep maintaining that code on my side.

So, with these transcripts I've been:

  - Creating YAML knowledge graphs
  - Summaries
  - Fixing transcription issues
  - Creating QA pairs
  - Identifying sections related to promoting a product or brand
  - Grading the output that it had generated on accuracy

So...it depends on your use case.

Not a surprise considering the recent events. Hope to see more information released in the meantime.

I am hoping that the extra time will mean that they get Custom GPTs to work even better.

The store will get flooded with lots of BS from people hoping to make quick bucks.

This is awesome for the movement to rename GPT to gipety and GPTS to gipeties ;)

If the surprise announce at dev day of custom GPTs* for money** was why Quora board member also running Poe got mad enough to oust Altman, perhaps this would be what Altman conceded to him in order to let Altman back (along with letting him keep his seat).

* Custom GPTs: https://techcrunch.com/2023/04/10/poes-ai-chatbot-app-now-le...

** Monetization: https://developer.poe.com/resources/creator-monetization

Adam

Too few people know this is the most relevant comment on this post.

Can you expand? I know nothing.

Adam d'Angelo[1] who is on the board is also the founder of a commercial AI startup called Poe which lots of people are claiming is a conflict of interest given openAI are moving into a space that seems adjacent/overlapping. Sam Altman said Adam is one of openAI's biggest customers and its helpful to have the customer perspective on the board.

[1] https://en.wikipedia.org/wiki/Adam_D'Angelo

How is Adam one of their biggest customers?

This is not where I originally read it, but it will do[1]:

"A person familiar with the situation said OpenAI's board appreciated having the perspective of a customer like Quora among its directors."

[1] https://tech.hindustantimes.com/tech/news/openai-director-ad...

Too soon

Reply on news.ycombinator.com