$20 is equivalent to what, 10,000,000 tokens? At ~750 words/1k tokens, that’s 7.5 million words per month, or roughly 250,000 words per day, 10,416 words per hour, 173 words per minute, every minute, 24/7.
I uh, do not have that big of a utilization need. It’s kind of weird to vastly overpay
I stood up a open source & login-free UI here: https://www.chatwithme.chat/
It accepts your API token and stores it in your browser. It does not have feature parity with ChatGPT but gives you the basics.
Unless I'm misunderstanding something, it does not sound like that much when every query you make carries several hundred words of prompt, context and "memory". If the input you type is a couple words, but has 1k extra words automatically prepended, then the limits turn into 10 queries per hour, or one per 6 minutes.
1. I ask Q1
2. API responds with A1
3. I ask Q2, but want it to preserve Q1 and A1 as context
Does Q2 just prefix the conversation like this?
„I previously asked {Q1}, to which you answered {A1}. {Q2}“
User: hello (previous prompt)
Bot: hi (previous response)
User: who are you? (new prompt)
Bot: (here it continues conversation)
I guess I will spend the next day playing around with the new API to figure it out.
[0] https://github.com/openai/openai-python/blob/main/chatml.md
You could try formatting it like
Question 1: ... Answer 1: ...
...
Question n: ... Answer n: ...
It makes you vulnerable to prompt injection, but for most cases this would probably work fine.
"The main input is the messages parameter. Messages must be an array of message objects, where each object has a role (either “system”, “user”, or “assistant”) and content (the content of the message). Conversations can be as short as 1 message or fill many pages."
"Including the conversation history helps when user instructions refer to prior messages. In the example above, the user’s final question of “Where was it played?” only makes sense in the context of the prior messages about the World Series of 2020. Because the models have no memory of past requests, all relevant information must be supplied via the conversation. If a conversation cannot fit within the model’s token limit, it will need to be shortened in some way."
So it looks like you pass in the history with each request.