Hacker News Reader

Comment by anaganisk | Hacker News Reader

Given that "Open" AI API is very pricey, Im waiting to see how economical is it to use this as it is heavily marketed by all these blog posts.

View on HN

For anyone else curious, the pricing according to gwern [1]:

    Explore: Free tier: 100K [BPE] tokens, Or, 3-month trial, Whichever comes first
    Create: $100/mo, 2M tokens/mo, 8 cents per additional 1k tokens
    Build: $400/mo, 10M tokens/mo, 6 cents per additional, 1k tokens
    Scale: Contact Us

Expensive but (unfortunately) pretty standard market price for cloud NLP stuff. The price is about $40 per million tokens.

For comparison, cloud translation costs per million characters range from $6 (Azure bulk pricing) to $20 (Google/AWS without bulk discounts). If we assume the average BPE token length is 5 characters, GPT-3 will cost about $8 per million characters.

[1] https://old.reddit.com/r/GPT3/comments/ikorgs/oa_api_prelimi...

In one of the Reddit comments about a service (PhilosopherAI.com) using the API outlining the possible total costs:

> That makes for 400 million tokens in 2 or 3 weeks, which puts me at like $4000/mo minimum

Oh dear, that means...

> Scale: Contact Us

So while they are using the API for free for now, by these metrics, they are least going to be soon having a running cost of at least $4K+/mo. That is an expensive toy. The same goes for AI Dungeon.

> App Store earnings will be used to cover OpenAI costs & develop new features. Thank you for your support.

Well I will be seeing a tsunami of complaints from OpenAI projects that they can't pay up for their "game changers" since they will be priced out quickly. OpenAI Wins, Everyone else loses.

If this is for simple projects, then I wouldn't dare build an entire startup based on someone else's API. OpenAI will win by default here.

I was thinking about Ai dungeon the other day. I can't imagine it really is sustainable with those 10$ subscriptions covering the free play folks too. What happens whenever they get featured on some bigger website?

AI Dungeon has already scaled up to over 1,000,000 players and managed costs. They have a whole article about it here:

https://medium.com/@aidungeon/how-we-scaled-ai-dungeon-2-to-...

An important note, as others have said, is that not every player gets the same model. Free players, as far as I'm aware, only get GPT-2 ("only" is a strong word, as up until GPT-3, GPT-2 was the state of the art in text generation and is still extremely impressive in its own right).

AI Dungeon free players only use GPT2 instead of the GPT3 API.

What is a token? One character? Or is it a currency?

A character, both input and output.

Not exactly, GPT-3 uses a variant of BPE [1], so one token can correspond to a character, an entire word or more, or anything in between. The paper [2] says a token corresponds to 0.7 words on average.

[1] https://en.wikipedia.org/wiki/Byte_pair_encoding

[2] https://arxiv.org/abs/2005.14165, page 24

Reply on news.ycombinator.com