GPT4 is up to 6 times more expensive than GPT3.5

HN's guidelines don't call for using the original title in every case; there are quite a few exceptions. Most likely your title broke one of those rules.

For example, it's certainly possible for a title to be linkbait even when it is backed by benchmarks. But I'd have to know which submission it was to answer in detail!

> Sometimes you have to abbreviate to keep a title under the character count.

Correct!

> Sometimes what is in the page's <title/> tag is not the same as the readable title on the article ... and [...] it is not unreasonable to use that.

Also quite correct!

> And something I've observed recently is titles on news articles being updated after they are published

Also entirely correct. (I don't know if it's pressure - sometimes they're just correcting things - but we don't really need to know why.)

> This doesn't justify wild re-writes ... but "original title" is not as clear cut as one might wish.

Absolutely right!

I think he means it was 30x more than gpt-3.5 is today.

The 32k tokens version should do fairly well on such codebase. I don’t know if it’s the one used in ChatGPT.

There are already some cool projects that help LLM go beyond the context window limitation and work with even larger codebases like https://github.com/jerryjliu/llama_index and https://github.com/hwchase17/langchain.

The only thing holding this back now is lack of enough context. That’s the big nut to crack. How do you hold enough information in memory at once?

I can't get it to run a single query today. Are you on the paying plan?

Are there truely top tier ML optimized chips? I would imagine the hardware space still having room to grow if just because most of the brainpower having been focused on x86, etc.

But also gpt-3.5-turbo is far worse than davinci-003.

> Although the question is: If ChatGPT can do almost anything GPT3.5 can do, then why is it priced almost 10x less?

ChatGPT is GPT-3.5; with specific fine tuning / chat-oriented training, and without customer fine-tuning (at least, currently) available. It’s the particular GPT-3.5 interface that OpenAI wants people to preferentially use, so the price structure artificially encourages this.

If I were to speculate, OpenAI is doing two types of optimizations:

1) Cost

2) Function

I suspect ChatGPT is 90% a cost-optimized version of GPT3, and 10% a function-enhanced one.

GPT3 was the first major public wow-able model, and I suspect it was not running on any sort of optimized infrastructure.

Didn't the CEO say during the GPT-4 demos that humans were involved to do the captions during all the images tests (using Be My Eyes volunteers) and that's why it took some time to show the Discord examples ?

If so, it could explain some costs indirectly.

It's not. I think you've perhaps misread "less than 3 times as expensive as" as "3 times less expensive as" or similar. It's more expensive but not more than 3x the cost.

At last they didn't call it freeai

The problem with the name isn't that they're charging. It's that they're implying that they're something in the ballpark of open source when they're not.

I think it was borderline defensible for them to not release the weights, but now they aren't even telling people the size of the model.

From their website:

>Our mission is to ensure that artificial general intelligence benefits all of humanity

I pay for ChatGPT Plus and have paid a fair bit in OpenAI API fees; I don't care that they are making money. The problem is the openwashing, not the monetization.

I feel like this is comparing apples and oranges a bit, depending on when you were a kid. Even within the GPU market, Nvidia wasn't a huge player for a while. It's not just that GPUs are more important today, but that Nvidia went from being a small player in the GPU market to essentially being the GPU market. At the same time, Intel's presence in the CPU market has never been lower. Most "computers" are now smartphones running ARM processors, data centers are using AArch64 more and more, there's a slowdown in businesses and consumers buying computers, and AMD is taking a huge share of the X64 market.

Likewise, Nvidia is priced at 102x earnings while Intel is priced at 14x earnings. Some of that might be that people are anticipating the GPU market to be where all the money is in the future, but a decent amount of that is pessimism about Intel's place in the market.

You're not even comparing the CPU market to the GPU market. You're comparing what was possibly the 3rd ranked GPU marker when you were a kid and Intel when you were a kid dominating with 90-95% of CPU sales with Nvidia being 90% of the GPU market's profits today and Intel being more of a bit player in the CPU market.

Some of Nvidia's size today is because they beat all the other graphics card companies (yes, ATI/AMD is still around, but Nvidia's marketshare is above 85%). Some of that comes from GPUs becoming a much bigger factor in many things like AI. But when comparing Intel and Nvidia today, Intel is still getting double the revenue and profits of Nvidia. This is despite the fact that most CPUs have moved away from their architecture, despite the fact that they've had at least half a decade of terrible engineering performance, despite AMD starting to kick their butt and taking serious marketshare on their architecture, etc.

I don't want to say that GPUs aren't important, but Nvidia being worth more than Intel probably says more about Nvidia executing well while Intel stumbles than anything else. If you were asking your father this in 1998, Nvidia was just releasing the RIVA TNT, 3dfx was still king, and Intel was so dominant it looked like they'd be unstoppable forever. Now you're looking at an Intel that doesn't even look like the king of their own architecture, has been having foundry issues for years, and an Nvidia that no other GPU maker has been able to touch. If another GPU maker beats Nvidia and Nvidia becomes smaller than Intel, does that mean that GPUs aren't important any more? No.

The problem isn't your conclusion that GPUs are important today, but the logic getting there. People bragging about things isn't necessarily a great barometer for future success. Lots of things are trendy and lots of companies come and go. Everyone was talking about Pogs and Beanie Babies at one point. That hasn't made them last. People were talking about smartphone/handheld devices since the 80s, but almost all of those companies aren't in that market today (Palm, Handspring, Blackberry, Microsoft). Instead, Apple won that market. Heck, you could have asked your father if 3dfx were bigger than Intel and seen them go bankrupt a few years later. Where's Matrox and S3? This is an anecdote that just feels like survivorship bias. Yep, you asked about the GPU maker that won the market and the CPU maker that would face big declines (and still Intel's revenue/profits are double Nvidia's).

*alarming

Genuinely curious as to why ChatGPT could not so I asked it.

Note the final bullet point. I posit, that if it is possible, it will happen (regardless of ethics), because the risk of not doing it (when your adversary will) outweighs the risk of doing it.

"You say an instance of the GPT cannot experiment with new architectures, but could it if a) it was modified by the developers and b) it was given access to sufficient computational resources?"

Response: "In theory, if a GPT model were to be modified by developers to include capabilities for analyzing, designing, and testing new architectures, and if it were given access to sufficient computational resources, it could potentially experiment with new architectures. This would involve incorporating additional machine learning and deep learning algorithms and methods, such as reinforcement learning, evolutionary algorithms, or neural architecture search, which can be used to explore new model designs.

However, there are a few caveats to consider:

- Developing such a system would require significant expertise in AI research, as well as an understanding of the specific problems that need to be addressed. - The GPT model is a complex system, and adding the ability to self-improve or create new architectures could make the model even more complex and challenging to manage. - The computational resources required to develop, test, and implement new architectures are substantial, and optimizing resource allocation would be a critical concern. - Ensuring the safety and ethical use of such a self-improving AI system would be an essential consideration, as it could potentially lead to unintended consequences or malicious use.

Although it is theoretically possible to create a GPT-like model with the ability to experiment with new architectures, it would require significant advancements in AI research, development, and ethical considerations.

Thanks for the reply. I'm not all that impressed by "what is"/information retrieval tasks, is it any better at "thought" tasks like explanatory reasoning and ideogensis? For instance - "How might an agoraphobic kleptomaniac satisfy both of their desires simultaneously?"

Those coffee puns are surprisingly good.

Bing Chat is more useful in many cases because it provides references that connect to it’s responses. I don’t like the mobile UI and that it doesn’t save your chats for you. Maybe I should try the web interface.

How are you trying it? I’m ChatGPT Plus subscriber but don’t see any change.

In what contexts is the difference the most stark?

Its not $ constrained.

The existing cloud providers don't have 10000 A100s lying around to ramp up at any time. There's a very limited amount of A100s and H100s on the market. TSMC is printing them like candy, but it still takes time to ramp up.

Azure also needs time to install them in datacenters. These A100s stacks probably produce monstrous amounts of heat compared to normal CPU racks.

No I'm just an idiot. Since the price is per token the actual ratio is more like 8x more (assuming you fill both contexts) which is plausibly the ratio between the costs.

Ah, OK. thanks for pointing that out.