I feel like this just killed a few small startups who were trying to offer more context.
Also, I pay for ChatGPT but I have none of the new features except for GPT4. Very frustrating.
Also, I pay for ChatGPT but I have none of the new features except for GPT4. Very frustrating.
The really odd thing is that I was given GPT-4 with browsing alpha enabled - for a single session last week.
As soon as I reloaded the page, it was gone. Since then the picture has reverted back to the above.
Twitter has become a bit painful to read these days, with all the AI influencers posting about what GPT-4 and plugins, code interpreter etc. can do.
Once I filled that in I got access within a few days.
https://openai.com/waitlist/plugins
If you are the person to say: "I am a developer and want to build a plugin"
Then it is likely you missed the option to request which plugins you want access to.
I stopped paying the pro because without plugins it didnt do that much tbh
52/1.92 = 27 416/1.92 = 217
So using GPT-4 with 32k tokens, 27 times per hour, or 217 times per day, in terms of cost, is approximately the equivalent of another dev
Not that it matters for the calculation, but i wonder how long such a request (ingesting 32k tokens and responding with a similar amount) would take.
At the speed of regular ChatGPT take would take a good while.
At the upper bound, this would be $2 * 3 * 60 * 4 = $1440 a day.
Thankfully, I am using retriever-augmentation and context stuffing into the base 4k model, so costs are manageable.
The 32k context model cannot be deployed into a production app at this pricing as a more capable drop-in replacement for shorter-context models.
Care you elaborate? This sounds very interesting & useful. Just anything about the setup and implementation would be super helpful.
I also find it strange they don't contrast gpt4 and gpt3.5
And don't forget that all the LLaMA-based models only have 2K context size. It's good enough for random chat, but you quickly bump into it for any sort of complicated task solving or writing code. Increasing this to 4K - like GPT-3.5 has - would require significantly more RAM for the same model size.
For things where correctness matters, the majority of cost will still come from humans who are in charge of ensuring correctness.
The latter is definitely the cheapest option; updates are trivial.
GPT-4 seems to show that linear algebra definitely can do the job, but training is so expensive and the model gets so huge and inflexible.
It seems like having fixed format vectors of knowledge that the model can use-- denser and more precise than just incorporating tool results as tokens like OpenAI's plugin approach-- is a path forward towards extensibility and online learning.
There may be a middle ground between these two approaches though. If every query used the same prompt prefix (because you only update the codebase + docs occasionally) then you could put it into the model once and cache the keys and values from the attention heads. I wonder if OpenAI does this with whatever prefix they use for ChatGPT?
I'm pretty sure they're using a 4k GPT-4 model for ChatGPT Plus, even though they only announced 8k and 32k... It can't handle more than 4k of tokens (actually a little below that, starts ignoring your last few sentences if you get close). If you check developer tools, the request to an API /models endpoint says the limit for GPT-4 is 4096. It's very unfortunate.
Another thing that annoys me is how most updates don't get a changelog entry. For whatever reason, they keep little secrets like that.
Every time I see a company act like this, more responsive and truly open competition eventually eats their lunch.
The initial blog post was only just over a month ago, and it was announcing alpha access for a few users and developers:
> Today, we will begin extending plugin alpha access to users and developers from our waitlist. While we will initially prioritize a small number of developers and ChatGPT Plus users, we plan to roll out larger-scale access over time.
https://openai.com/blog/chatgpt-plugins
We are literally 1 month into the alpha of plugins.
The developer livestream was on March 14th: https://www.youtube.com/live/outcGtbnMuQ?feature=share.
The time since GPT-4 already feels something like 6 months. So far I'm perpetually feeling behind.
I struggle to keep up and all I need to do is understand developments well enough to simplify them in to palatable morsels for my tech skeptic colleagues in politics and non profits.
Challenging because they have a form of technology PTSD. when they hear "new technology" nft's of monkeys with 6 digit prices and peter thiel's yacht flash before their eyes and they see red.
And I can't really blame them, the rhetoric around crypto was enough to sour most non techies (in my little corner of lefty politics anyway) against the idea that any tech advancement is noteworthy. One of the first more serious individuals in politics to hear me out did so because "i sounded like one of the early linux proselytizers" lol.
Completely agree how time has slowed. I rotate between absolute giddy anticipation at our future thanks to the tech and nihilistic doomerism. Even as a hobbyist though I knew to take this seriously since I saw robert miles talk about gpt 2 in 2017(?) and note there's zero sign of these things plateauing in ability simply by ramping up parameter count.
I've gone on long enough but that live stream felt like the intro to a sci fi movie at points. Can't wait to have multi modal and plugins rolled out.
I expect that in the next 5yrs developer workflows will completely change based on all the LLM stuff.
I think it's always difficult to tell if new tech is just hype or will have real impact, but it really feels to me like LLMs will have real impact. Maybe not as much as they are being hyped, but definitely legit impact. There's a possibility of even greater impact than the hype as well.
It’s very slow, almost 10X slower than ChatGPT
It’s integration is bad. For most plugins it doesn’t do anything smart with its API call. For example if I ask “Nearest cheap International flight”, it literally goes to Kayak and searches Nearest Cheap International Flight, if Kayak can’t handle that query, GPT can’t either.
The only plug-in with good integration is Wolfram and it makes so many syntax errors calling Wolfram that it’s thrash. Often it just syntax errors out for half my queries
I wouldn’t have minded if they spent a few more months internally testing plug-ins before rolling it out to me, seeing it’s current state. The annoying thing is the chat website automatically starts at plugins mode which is borderline unusable. So every time I have to click on the drop-down and then choose ChatGPT or GPT4.
For code, I use phind.com.
Those startups killed themselves. A 32K context was advertised as a feature to be rolled out the same day GPT-4 came out.
Also - what startups are getting even remotely close to 32K context at GPT-4’s parameter count? All I’ve seen is attempts to use KNN over a database to artificially improve long term recall.
(Also UI probably tanks too. I dread what the OpenAI Playground will do when you start actually using 32k model for real, like throwing a 15k token long prompt at it. ChatGPT UI has no chance.)
Until they cut down the cost then they should worry yeah