GPT-3 pricing for API usage announced to beta users to go in effect from October | Hacker News Reader

GPT-3 pricing for API usage announced to beta users to go in effect from October

reddit.com

93 pointshardmaru45 comments

Open articleView on HN

Beginning October 1st, 2020

Free tier: 100K tokens

$100/mo tier: 2M tokens, $0.08 per additional 100K tokens

$400/mo tier: 10M tokens, $0.06 per additional 100K tokens

It sounds like the free tier is more of a trial with 100k tokens total, not per month

And only 3 months to use said 100k tokens.

The 3 month window is the killer. free tiers are there to give time for developers to learn an API, or to prototype something nights/weekends.

Correction: it's $0.08 per additional 1K tokens. That is $8 for 100K tokens.

If I understand the pricing correctly, that seems prohibitive. The models go up to 2k tokens of input, so wouldn't that mean that it costs $.16 per output token?

I'm confused as well, what's a 'token' and how many do I need?

Some FAQ items:

What does 2M tokens equal in terms of number of documents/books/etc?

This is roughly equivalent to 3,000 pages of text. As a point of reference, Shakespeare’s entire collection is ~900,000 words or 1.2M tokens.

Will the API be general public access starting 10/1?

No, we will still be in limited private beta.

How are the number of tokens per each subscription tier calculated?

The number of tokens per tier includes both the prompt and completion tokens.

How are tokens differentiated across engines?

These token limits assume all tokens are generated by davinci. We will be sharing a reference legend for other engines soon.

What will fine-tuning cost? Is it offered as part of this pricing?

Fine-tuning is currently only available for the Scale pricing tier.

Like, this doesn't tell me how much project X or project Y will really cost.

I hope there will be resellers. There is room for a lot of markup in the small volume.

I think these volumes are relatively small. Bear in mind that this is tokens, not API calls. Depending on how much input text or output text there is a single call could easily be using 100s of tokens.

Resellers sounds like something that would be explicitly against the terms of service.

Anyone selling access to a GPT-3 based product will be a reseller, strictly speaking.

So, providing no value and simply making money off of less knowledgeable users?

Nah, I hope there won't be any resellers.

well, the value would be allowing other users to purchase smaller amounts.

Maybe I have some hobby app that still sometimes goes over the 100k free tier limit, I might be paying $10/month to get 50k extra but I'm not willing to pay $100 for 2M when I'm never going to use 90% of that.

There is no free tier. You get 100k token that you can use for free and you have 3 months to use them.

Not looking forward to undetectable spam emails. Aaahhh.

On the other hand, maybe Interseller can integrate this and I can send slightly different emails in a sequence automatically.

Spam problem is absolutely miniscule in comparison with the forthcoming. GPT-N running in medium-sized datacenter would be able to outargue 90% of people on any topic. Imagine each person in the world been assigned his own personal team of propaganda agents. The defence would be an independent ID system. Something historically done by governments only, with a sizeable infrastructure.

I feel like the best way to filter spam is using information content. As it turns out, nobody is handing out $1,000,000 for free no matter how nicely they write the email.

Isn't that the whole point of gating it behind an API?

There will always be a spammer with connections to a large company or university.

It's not only having the connection, it's that they are presumably able to audit your usage and reserve the right to cut you off if they find out that you're using it for something nefarious.

OpenAI but with propietary code... They don't deserve that name

Training those models are extremely expensive and it's gonna find a way to recover that cost through one way or another...

The issue here isn't the cost to run the service, it's the name.

An analogy would be a restaurant called "free lunch" that charged money for lunch because food supplies are expensive.

I thought the model was publicly shown? You can take that and do your own training and make your own API, right?

No. GPT-2 is available to the public, at least up to so many millions of parameters. GPT-3 has so far been released exclusively to a small number of beta testers with strict rules on what you can do with it.

Bear with me, I am a ML amateur. What is this then? https://arxiv.org/abs/2005.14165

Could someone clarify what exactly are those tokens ?

Length of given text after encoding it with their BPE. In general, you can expect it to be 1.2x-2x the len(words) in your text.

The Pricing FAQ notes that it's, on average, 4 characters, which IMO is fair.

Super common punctuation is its own token, however.

Screenshot of the pricing grid: https://twitter.com/sonnylazuardi/status/1301045589619040256

is there any non-malignent use case for gpt-3?

Lots of them: https://github.com/elyase/awesome-gpt3

I used talktotransformer.com to write stories for myself. Some of them were great inspiration material. The bot's pretty good with ideas.

I agree, GPT-2 makes an excellent writing companion. I actually developed my own version of talktotransformer.com to introduce some features that were missing from the public demo, including:

* support for custom/fine-tuned/international GPT-2 models. Use any text generation model you want!

* support for multiple outputs. Generate as many suggestions as you want!

* support for customizing the prediction length. Can't think of that next word? You can effectively 'poll' the model by reducing the window size and increasing the number of suggestions: the most 'sensible' (or safe) choice usually appears most often. Or, go wild and let GPT-2 write the entirety of your boring, corporate email for you.

* a more interactive user experience. No need to submit a blocking POST request and wait for the server to respond; just type '...' to (async) fetch a new batch of suggestions. The interface will update as soon as those predictions become available.

Here's a video of the application brainstorming some new features for itself: https://streamable.com/jij614

I didn't realize talktotransformer.com had been taken down. I hadn't been in a hurry to publish the code, mainly because it's going to be a little tricky to distribute (you might have noticed in the video that the GPT-2 model is loaded into a Google Colab notebook that accepts GET requests, then queried remotely), but if people are interested I can absolutely give it a shot.

That is very cool, my dude. Someone else also built an AI writing assistant like this that I tried. It was really quite good.

"This demo (talktotransformer) is turned off for the moment." It appears to have been replaced by https://inferkit.com/ . I don't know how the usage limits for inferkit compare to talktotransformer. The pricing page can be found here https://inferkit.com/pricing .

TalkToTransformers was free when the hardware is really, really expensive to scale.

thanks for posting:

They are using GPT-2 if I am right, will they use GPT-3 in the coming weeks? Btw, I would love to know any other similar services that can let you input a sentence or words to generate any length readable contents.

Thank you!

All the good ones would require the actual model and not this shitty API

My guess it's will be very useful for helping weak writers write better texts. Like the user tells the computer what they want to convey through a letter/email/text, the computer gives them like 3 example sentences that could work and the user picks what is most applicable. There is probably going to be a booming business of AI assisted writing.

AI Dungeon is a pretty great use case. I'm not sure if it'll survive this pricing, though.

What's the best way to gain access as an european?

I thought this was openai not AIASS (AI as a service)

I propose a new Turing test:

API Fees for GPT3|N < revenue earned from Amazon Turk Hits.

It would be really cool if the GPT-3 pricing page was generated with GPT-3 :)

Reply on news.ycombinator.com