This is one hell of a loss leader to keep people from using competitors, especially since it appears that it’s not being used to upsell to a paid offering.
This can’t be sustainable even with all the inference optimizations in the world.
This can’t be sustainable even with all the inference optimizations in the world.
I believe OpenAI has elected to eat the compute cost in order to teach the model faster to stay ahead of competitors. How much are you willing to pay as the robot gets better faster? Everyone fighting over steepness of the hockey stick trajectory.
Anyone can buy GPUs, you can’t buy human attribution training. You need human prompt and response cycle for that.
Also, very few people know as much as sama about making startups grow, so ...
PS. I'm not even a fan of "Open"AI, but it is what it is.
Unfortunately, Claude.ai is only available in certain regions right now. We're working hard to expand to other regions soon.
"New model coming soonish" (Sure, so where is it?)
"GPT-4 kind of sucks" (Altman seemed to like it better before Athropic beat it)
"[To Lex Fridman:] We don't want our next model to be shockingly good" (Really ?)
"Microsoft/OpenAI building $100B StarGate supercomputer" (Convenient timing for rumor, after Anthopic's partner Amazon already announced $100B+ plans)
"ChatGPT is free" (Thanks Claude!)
So, in same time OpenAI have gone from GPT-3 to GPT-4, Anthropic have gone from startup to Claude-1 to Claude-2 to Claude-3 which beats GPT-4 !
It's not just Anthropic having three releases in time it took OpenAI to have one, but also that they did so from a standing start in terms of developers, infrastructure, training data, etc. OpenAI had everything in place as they continued from GPT-3 to GPT-4.
1. They state upfront the salary expectations for all their positions 2. The salaries offered are quite high
and I immediately decided this was probably the company to bet on, just by virtue of them probably being able to attract and retain the best talent, and thus engineer the best product.
I contrasted it with so many startups I've seen and worked at that try their damnedest to underpay everyone, thus their engineers were mediocre and their product was built like trash.
But yes it also helps attract talent.
I have also personally found that minimizing the context window gives the best results for what I want. It seems like extra context hurts as often as it helps.
As much as I hate to admit it too but there is a small part of me that feels like chatGPT4 is my friend and giving up access to my friend to save $20 a month is unthinkable. That is why Claude needs to be quite a big upgrade to get me to pay for both for a time.
However once I figure out a way to download all my chats from Chatgpt, I think Claude' 200k context window may entice me to rethink my Chatgpt subscription.
Yes, really.
They're strongly influenced by Yudkowsky constantly telling everyone who will listen that we only get one chance to make a friendly ASI, and that we don't have the faintest idea what friendly even means yet.
While you may disagree with, perhaps even mock, Yudkowsky — FWIW, I am significantly more optimistic than Yudkowsky on several different axies of AI safety, so while his P(doom) is close to 1, mine is around 0.05 — this is consistent with their initial attempt to not release the GPT-2 weights at all, to experiment with RLHF in the first place, to red-team GPT-4 before release, asking for models at or above GPT-4 level to be restricted by law, and their "superalignment" project: https://openai.com/blog/introducing-superalignment
If OpenAI produces a model which "shocks" people with how good it is, that drives the exact race dynamics which they (and many of the people they respect) have repeatedly said would be bad.
re p(doom): the latest slatestarcodex[0] has a great little blurb about the difficulties of applying bayes to hard problems because there's too many different underlying priors which perturb the final number, so you end up fudging it until it matches your intuition.
[0] https://www.astralcodexten.com/p/practically-a-book-review-r...
Edit: I can't quite put this into a coherent form, but this vibes with Gell-Mann amnesia, with the way LLMs are dismissed, and with how G W Bush was seen (in the UK at least).
Ironically, a similar (though not identical) point about Bayes was made by one of the characters in HPMOR…
[0] Also the whole bit of HPMOR in Azkaban grated on me for some reason, and I also think Yudkowsky re-designed Dementors due to wildly failing to understand how depression works; however I'm now getting wildly side-tracked…
Oh, and in case you were wondering, my long-term habit of footnotes is somewhat inspired by a different fantasy author, one who put the "anthro" into "anthropomorphic personification of death". I miss Pratchett.
In case you still don't believe me, you are welcome to hop onto your favorite fan fiction site, such as AO3 [0], and search for stories over [large k] words.
[0] https://archiveofourown.org/works/search?work_search%5Bquery...
Also, I didn't say "correlated with intelligence", what I said was more of a cut-off threshold — asserting that one cannot be an actual moron given writing coherently on that scale is more of a boolean than a correlation.
I do need to write more (my own attempt at a novel has been stalled at "why can't I ever be satisfied with how I lead into the dramatic showdown!" for some years now, as none of my attempts have pleased me once written); but as for reading? Well, if you think my taste must result from insufficient breadth and depth, I must wonder what you think of Arthur C. Clarke, Francis Bacon, Larry Niven, Alexandre Dumas, Adrian Tchaikovsky, Neal Stephenson, Robert Heinlein, Alastair Reynolds, Isaac Asimov, Carl Jung, … I'm going to stop there rather than enumerate my whole library, but I can critique each in a different way without resorting to calling them playground insults, even the ones I dislike.
But I will say it was interesting to contrast Chris Hadfield's "An Astronaut's Guide to Life on Earth" with Richard Wiseman's "Shoot for the Moon" — or H. G. Wells with Jules Verne.
If we ignore that it's a rewriting of JKR's 7 novel series, which gives it a certain amount of coherency, Yudkowsky violates almost every writing guideline in a bad way. In fact, I could probably write an infinitely long coherent essay describing the ways HPMOR violates a reader's mind. It would be easy given almost 700K source material.
But to point at some gaping holes, the plot has no pacing, and the entire story is a badly written self insert where the mc goes around and "fixes" JKR's plot-holes by writing themself into a corner.
The solution to this, is, of course, to write another 20K words of expecto patronum, dispel the plothole with more rationalist bullshit.
Your average fanfic is probably better written.
I infer you favour Blaise Pascal: "I'm sorry I wrote you such a long letter; I didn't have time to write a short one."
his writing may be a little cringe at times and not anywhere near the prestige of "real writers" but it's perfectly entertaining for his intended audience
These smaller models are a relatively cheap to run, especially in high batches.
I'm sure there will also be aggressive rate limits.
like 6-7 months ago there was a gpt4 version that was really good, it could understand context and stuff extremly well, but it just went downhill from there. i wont pay for current chatgpt 4 anymore
This is why I'm excited for the growth of local model capabilities. I can much more reasonably expect that the model has not degraded and that it is using the full hardware capabilities it has been granted.
They apparently constrained this publicly available version, with no gpt-4 or dall-e, and a more limited range of topics and uses than if you sign up.
They do explicitly recommend upgrading:
We’ve also introduced additional content safeguards for this experience, such as blocking prompts and generations in a wider range of categories.
There are many benefits to creating an account including the ability to save and review your chat history, share chats, and unlock additional features like voice conversations and custom instructions.
For anyone that has been curious about AI’s potential but didn’t want to go through the steps to set-up an account, start using ChatGPT today.