ChatGPT "All Tools" System omni-prompt

The "All Tools" omni-prompt takes a whopping 2,756 tokens, but it's also using the GPT-4 32k model, with a 32,767 token context window.

It also reduces default DALL•E generations to one 'caption' (with a conflicting instruction later to generate 2 images).

Custom Instructions are going to need some work, since all this stuff comes before CIs.

The other thing that I found interesting: the built-in RAG accepts a limited number of file types, and uses the same `tools` middleware that the `browser` tool uses. It seems that it renders those files directly in their headless browser container, and captures text from it exactly the way it does when using a the Browse with Bing.

It also stores those files in the `/mnt/data` sandbox used by the `python` tool, making the uploads automatically ephemeral.

File types supported:

    "text/plain",
    "application/pdf",
    "text/html",
    "text/x-tex",
    "application/vnd.openxmlformats-officedocument.presentationml.presentation",
    "application/json",
    "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
    "text/markdown"

There are three middlewares (for lack of a better term) used by ChatGPT to talk to external tools.

- `tools` (browser)

- `tools2` (python/jupyter)

- `tools3` (plugins)

It's weird that DALL•E clearly uses plugins, but the model metadata endpoint [0] doesn't declare that.

[0]: https://github.com/spdustin/ChatGPT-AutoExpert/blob/main/_sy...