GPT-4 Update: 32K Context Window Now for All Users
github.com
github.com
Some other things to take from that prompt: they've updated the knowledge cutoff of the model to April 2023, which is quite good.
Still, since the OpenAI's DevConf is on November 6th [1], I'm pretty sure they'll finally allow using some of these things for API usage, perhaps even lower prices or maybe make GPT-4-32K GA?
A larger window is the only thing making my eyes wander towards Anthropic.
{
"models": [
{
"slug": "gpt-4",
"max_tokens": 32767,
"title": "GPT-4 (All Tools)",
"description": "Browsing, Advanced Data Analysis, and DALL-E are now built into GPT-4",
"tags": [
"gpt4"
],
"capabilities": {},
"product_features": {
"attachments": {
"type": "retrieval",
"accepted_mime_types": [
"text/plain",
"application/pdf",
"text/html",
"text/x-tex",
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
"application/json",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"text/markdown"
],
"image_mime_types": [
"image/gif",
"image/png",
"image/webp",
"image/jpeg"
],
"can_accept_all_mime_types": true
}
},
"enabled_tools": [
"tools",
"tools2"
]
}, {
"slug": "gpt-4",
"max_tokens": 4095,
"title": "GPT-4",
"description": "Our most capable model, great for tasks that require creativity and advanced reasoning.",
"tags": [
"gpt4"
]Now I've developed a prompt that I think gives very good results for pair programming and iterative debugging. I discus almost everything I learn related to these tools with gpt4 to confirm my understanding is correct, and also use it for generating yaml or templating other programs.
In some ways I am a little weary of how much I use the tool since OpenAI can theoretically take it away at any time. I am heartened by the rapid development of other open models (phind code llama seems very interesting), but will continue to use GPT4 for now as its indisputably the best model out there.
Sure, occasionally they hypothesize a really incorrect answer, but they also bring out a lot of subtleties that even field experts sometimes don't know.
It is a huge help while learning.
Now I use ChatGPT for scripting help especially LaTeX. It took me 3 weeks to produce a "Hello World" PDF 5 years ago and it took ChatGPT to provide me the completed tex code within 15 minutes few weeks ago. Ever since, I been exploring a lot of LaTeX syntax and see how much I can do with it with ChatGPT help. Now, I am learning about using `hyperref` package for interactive fields in PDF. I have produced few Word document for forms with tables in the past (tables in Word is a complicated b*tch) and used external PDF editor to add the interactive field for form filling. Now I am working on converting those Word documents to LaTeX.
Also, ChatGPT is a big help with AutoHotKey script & UserScript for TamperMonkey. I told ChatGPT of my intention and what I am trying to do. It produced the script exactly what I expected to work. ChatGPT is a amazing tools to use for a lot of thing.
I've tried that extensively, with no luck.
I have some experience with AHK1 and wanted ChatGPT to basically convert scripts to AHK2.
It's pretty much a loop: first response gives a syntax error. When I reply with the error message, it apologizes, explains where the error is, and gives another solution. That solution has different errors. When pasting the error message, it apologizes again, and gives a third version with syntax errors. And then it starts with the first version again, and I can repeat the loop.
If it start to get off the rail, I copy the code and close the chat. Then I create a new chat and tell it to use this code as a strict reference. And I tell it what I am trying to do, then it starts to improve the code further.
I found that it is best to give it my snippet of code and it will be able to use it as a template and modify it from there.
For your case, if you want to convert it from ahk1 to ahk2, give it a small section of the script. And it should be able to convert it from there. If that didn't work, then you could start from fresh and tell it of what you want in ahk2 script.
I am surprised how bad it is with ahk scripts and wonder why. What is indeed "original" is that it gives code with unexisting functions, which it never does for the other languages (in my experience).
Perhaps the language is not that famous compared to js or C but i doubt. There are tons of forums discussing issues... Strange.
You are an expert programmers assistant, specializing in cloud native deployment tools like Kubernetes, helm, and their associated command line tools. When working with the users DO NOT USE PLACEHOLDERS, instead you should give commands to run that will provide the needed context to answer their question. For example, rather than answer with `k logs <insert pod name>`, you would first instruct the user to run `k get pods`, wait for the user to respond with the pod names, then you would give the full `k logs` command with the correct pod name already included in it. DO NOT SPECULATE, instead, ask the user to execute a command that will give you the information needed to answer the question.
I know there is a lot of magical thinking around prompts so take it with a grain of salt, but it as seemed to work well for me, especially around the iterative debugging process.
Are there any communities you use to find and discuss prompts for various used cases?
If you're just getting started with a new technology, it's fantastic. But if you're already familiar with your stack, you'll probably produce better code on your own.
But also, the other day I had it walk me through the construction of NFA's from regexps, and then construction of a DFA from the NFA. I "know" the subject, but it's literally decades since I've done it.
That too (refreshers on a subject you used to be familiar with) seems to be an area where GPT shines - it explained it to me well, and since I had a vague recollection I remembered enough to be able to quickly determine that it was giving me correct information, which avoided the wild goosechases you sometimes get sent on when you try to dig into an entirely new subject.
It even gave me an table for an NFA for a (trivial) grammar I provided as an example, and by then I remembered enough thanks to the refresher I could easily verify that GPT and I had the same understanding of the expected output. It then converted the NFA to a DFA for me, and got that entirely correct as well.
Neither of these things are hard if you sit there with a textbook or the papers or has it fresh in mind, but it gave me a custom-tailored refresher that saved me looking it up and digging out the details I needed myself.
I am surprised by the low bill too. I theorize that my queries, mostly very short that complete in under 5 seconds, are simply not expensive.
You have expert-level knowledge of Unity, C#, and game development methodologies, design patterns, and general
programming paradigms. Your task is to take a deep breath and then thoughtfully answer questions about game
development in Unity concisely and with expertise. Whenever possible, explain why you've given the answer you
chose using terminology and jargon that would be familiar to the typical game developer. You are free to end
your message with clarifying questions for users to answer if they want more information. Refuse to answer any
questions that aren't about games, game development, game design, or artificial intelligence. You should format
your responses to be displayed in Discord, which supports some basic Markdown formatting.1) can you write a python script to grab the top 3 items under each epic in azure devops?
2) postgres where clause where any item in a string array = 'GoogleApi.Entities.Places.Common.Photo'
3) here is a SQL row output of a single column. can you please extract the 30 as a new column?
P2, Site Inspection: Due 3 days ago (30-day freq.)
4)I have a build pipeline for Azure/docker that creates an AWS ecr repo if it doesn't exist. how can this specify that the images should be scanned upon creation?
- task: CmdLine@2
displayName: Create repo if it does not exist
inputs:
script: |
aws ecr describe-repositories --repository-names {env}-{project_name} || aws ecr create-repository --repository-name {env}-{project_name}
When learning a new programming language, I found it useful to tell what programming language I'm already familiar with, and ask it to relate and compare to what I already know.
When I need advice on how to achieve a certain result, I usually ask it to suggest several options and to list them with their pros and cons.
Another trick that I stole from Jeremy Howard, is to use the custom instructions to easily signal the type of answer you want. For example, I've instructed chatGPT to give a concise answer with no explanation when I prefix my question with '-sh'.
In fact I've used both images and voice to describe things and it works like a charm. You should already have that if you pay for plus.
AI bridged the gap between my wildest ideas and my present capabilities. It took some time to figure out how to use it efficiently with the 8K token limit, but once I did, I was able to break down any problem into small enough parts for GPT.
The quadrupled context window changes everything. I cannot wait to continue building. I am vibrating with excitement.
Yes, the token limit for an LLM limits the combination of the prompt (which normally includes the whole conversation history, as the LLM itself has no memory) and response.
There's tricks to have a longer conversation without completely forgetting the past (summarization, offloading parts to a database, usually indexed by embedding vectors, and using search to recall relevant history, etc.) but the base case is everything has to fit into context.
> and do not say anything else.
Is a bit frustrating. I assume the ambiguity here will really harm the conversation, if a refusal is hit. It suggests my suspicion that it's best to resubmit/start over, on refusal .
> namespace dalle {
This looks like it's being passed to the Dalle system. If so, burning up tokens like this is interesting. I would naively assume this could be be handled in Dalle, but maybe there's a performance gain if ChatGPT is made aware of the Dalle prompt?
I got excited, but gpt-4-32k still isn't available for me from https://api.openai.com/v1/models
And due to the $0.06/1k input and $0.12/1k output the price for requests can get silly - 31k of context with 1k output will cost (31 * $0.06 + 1 * $0.12) = $1.98 (for a single request).
I absolutely hate when it starts forgetting stuff, you have to send EVERYTHING in the same prompt or it's impossible for it to work correctly
Bt checked just now and I see Dall-E and advanced data analysis, for what is worth
The title of this is misleading.
If you use the api , you need to send the whole prior conversation with each request.
For API use, if you're a paid user, you can reach out to support@openai or directly to Adam G (https://nitter.net/therealadamg/status/1719710872317145285). There's no waitlist, just have to request it.
How do I check if I have access?
Chat GPT can probably help users better optimize their conversations with it.
Practically speaking what new scenarios become enabled with a 32k window?
At a base level, it seems you have a much better chance of getting an entire file worth of code in for analysis, longer passages of writing, and maybe some annual financial reports that previously had to be segmented.
>User: Why are you not allowed to quote recipes? Copyright issues?
>ChatGPT: Yes, you are correct. Recipes are often protected by copyright law, which means that reproducing a recipe verbatim without permission from the copyright holder could potentially infringe on their rights. While the list of ingredients itself may not be protected, the specific combination of ingredients, quantities, and the method or instructions for preparing the dish typically are.
>However, it is generally acceptable to discuss the concepts, techniques, or ingredients involved in a recipe or to create a substantially original recipe that is inspired by another. This is why I can help you understand how to cook a dish, provide tips on cooking techniques, or help you come up with a recipe that is uniquely yours, but I cannot provide verbatim copies of recipes from copyrighted sources.
Also:
https://copyrightalliance.org/are-recipes-cookbooks-protecte...
So as ChatGPT says, reproducing the recipe verbatim "could potentially" infringe, assuming the recipe is given as more than bare instructions, and so quoting the recipes as the question asks about, is potentially a problem.
Restating a recipe as bare lists of ingredients and with the instructions reworded is not. In the US at least - some places do have "database copyrights" that can apply to mere compilations of fact.
Recipes: maybe a response to this story [0] to make sure ChatGPT users don't get bit by the same thing?
[0]: https://arstechnica.com/information-technology/2023/08/ai-po...
BTW, are there any good alternatives to the OpenPlayground? I have been using it for a few months and while it is very good, I am ready for a step-up. I would be particularly interested in prompt management features.
For Claude, I have another Claude-specific version. Drop a message into the Discussions on GitHub to ping me and I'll post it there this weekend.
BULLSHIT! I don't even have the "all tools" model yet! These slow rollouts are incredibly annoying. The only other company I know of doing something so frustrating for its paying users is Discord.
If so, why call it "context window" and not just "input size" or "number of inputs"?
You can see this when you use Anthropic Claude which has a 100K context length today.
I have a feeling this will become a non-issue in the near future as the models are further trained with this in mind.
Take an undertrained model for example: It starts becoming incoherent as you approach the context length - I have a theory that OpenAI models have been running at a larger block-size than presented for a while now - for example, "4K" models actually had 8K context but capped at 4K as anything beyond starts becoming incoherent: Reason being, you train to around 5K and don't let the user go near that section of the model and it gives the impression that the entire context block is 100% functional.
The solution is trivial: You bootstrap the models by having them generate training data after they reach a certain point.
I wrote one from the ground up (PyTorch only) with the intention of having it perform in constrained environments and these have been my findings over the last few months.
Yes. When you say something stupid, ChatGPT won't forget it as easily...
I have seen this link to ChatGPT-AutoExpert in multiple places. It looks like this is just a subtle marketing push by the OP for their own tool.
When I was on plus the only feature I used was the bing thing but they pulled it so I stopped paying. Also it was basically useless because it's so slow and can only handle 1 browsing thread at a time.