Show HN: Python package for interfacing with ChatGPT with minimized complexity | Hacker News Reader

Show HN: Python package for interfacing with ChatGPT with minimized complexity

github.com

129 pointsminimaxir35 comments

Nice!

Totally agree with the project goals, it seems too many other packages are created by people who are researchers (or enthusiasts) first and software developers second, and it shows.

I see you're using Pydantic. I've recently been playing with using pydantic to implement chatgpt functions, making it a bit easier to define functions (tools) with more control over the attributes, like this:

    class SearchWeb(pydantic.BaseModel):
        """
        Docstring description to help GPT figure out what this does, like functions in your library.
        """
        query: str = pydantic.Field(description="More info so GPT understands how to use this param")

    def handle(self):
        # my wrapper will call this to implement the tool after the arguments are parsed
        # at this point you can be sure self.query is correct and has passed any validation you might have

It's definitely more verbose than the function definitions you have now, but you get schema definition for free, and is more strict about option parsing. It also makes it easy to throw errors back at GPT if it hallucinated some parameters incorrectly.

...aaaanyways, great work there, I'll be following the progress!

There was another post today about using Pydantic for function enabled completions: https://github.com/jxnl/openai_function_call

I whipped up an example doing something similar last Friday using a decorator, inspect, ast and __doc__ usage: https://gist.github.com/kordless/7d306b0646bf0b56c44ebca2b8e.... The example pulls top results from Algoia's HN search and then chains them into another prompt for GPT-X. The blog post is here: https://www.featurebase.com/blog/function-integration-in-ope...

Currently integrating this approach into PythonGPT[1], which will build a function on the fly, extract the method info, then call the code in exec(). I would label it "very dangerous"...

[1] https://github.com/FeatureBaseDB/PythonGPT

> There was another post today about using Pydantic for function enabled completions

That post let me know that pydantic's schema() function actually worked to produce a valid JSON schema, so I was able to optimize from there. (there may be a few optimizations still to be done: schema() also returns an unnecessary title field and I need to experiment if I need to remove it)

Ah, I see I've been ninja'd by about 17 hours: https://github.com/minimaxir/simpleaichat/releases/tag/v0.2.... :-D

I've spent a lot of time experimenting with schema: https://github.com/minimaxir/simpleaichat/blob/main/examples...

As a researcher what would the number one sin be that we commit? Is it something I could make the effort to improve upon?

Not a sin, I don't want to diminish in any way the awesome work that's being done!

It's just that these packages/libraries/frameworks are often in what I'd describe as "proof of concept" phase, not very developer-friendly (as in user-friendly for people wanting to try it out), missing docs, not handling errors, and not being written in a maintainable way.

So I think a "second generation" of tools/libraries, that are basically product-level quality, bringing on your work and focusing on those non-core-tech aspects of the experience, will be a next step to bring AI to the (developer) masses. Tools such as this package.

I hope you keep investing in this project- it’s a crowded field but I do think existing projects like Langchain and LLamaindex feel very bloated and beyond a refactor - sometimes we need fresh takes like this and start over.

I am: the reason I made simpleaichat is because I have a need for more controllable work with AI generation, and it was easier to create a package from scratch than to figure out how to hack LangChain.

Any chance your roadmap might include access to some local run open source models kind of like an oobabooga but with all your extra tooling baked in ?

Is interfacting with the GPT-4 API acting as if you are the ChatGPT UI against terms of service? I was under the impression it was and they had a bunch of methods to protect against this. They want you using ChatGPT for $20/mo and then paying for GPT-4 API usage separately per token from what I can tell.

You have to have an approved GPT-4 API key to use model="gpt-4". Nothing in this package circumvents it, which is what the free GPT-4 apps were doing.

I don't have a GPT-4 API key, but I can use GPT-4 in the OpenAI official ChatGPT UI, right?

The ChatGPT UI is not the API.

Does the ChatGPT UI use the GPT-4 API?

Why can I use the GPT-4 API through the ChatGPT UI, but not any other way?

Didn't find this in the README: does it handle network/OpenAI errors via retries? In particular the very common "model is overloaded at the moment".

It's fine if not, I can imagine you want to keep it as simple as possible and outsource it to the library user.

I was mixed about putting it in the base package but you can get the same behavior with the tenacity package.

Errors are so common with these endpoints, not having retry out of the box just makes it harder to hack on which I think is against the spirit?

On the other hand, automatic retires can be suprising, especially when they don't completely understand the returned error and retry on errors they shouldn't.

I just wasted a couple hours the other week due to langchainjs defaulting to retrying errors which were ultimately caused by timeouts and would never complete.

Yeah, I understand. Maybe it would be good to add a comment on it in the README, since most people will hit this problem.

Since this is async, can it automatically handle provided rate limits and batch queries appropriately?

Seems like everyone has to roll their own on this and it’s much nicer to have smooth tooling for it.

The underlying library for both sync and async is httpx (https://www.python-httpx.org/) which may be limited from the HTTP Client perspective but it may be possible to add rate limiting at a Session level.

Nice work!

Is there some JavaScript/Typescript alternatives to Langchain?

Langchain ? - https://js.langchain.com/docs/

Are we anywhere close to being able to give a whole codebase to GPT-4 and ask various things about it?

My open source CLI tool can do this, as well as make changes to the code. Here's a chat transcript that shows how you can use it to explore and then modify a public github repo for a js game:

https://aider.chat/examples/2048-game.html

Here a link to aider with more info:

https://github.com/paul-gauthier/aider

Thanks for sharing. Looks very promising.

I take it that asking things about the repository with gpt3.5-turbo-16k is not possible without adding each file separately?

Just to test I generated ctags for my smallish Rust project and noticed it comes to around 10k tokens according to https://platform.openai.com/tokenizer.

I don't have GPT-4 API access yet but it appears it would get quite expensive with the 32k context.

Correct, right now aider does not provide the ctags repo map to any gpt-3.5 models. I still need to evaluate if those models can actually understand and use the map effectively.

Improving and deepening support for the 3.5 models is probably my top priority next effort. Previously, they were barely useful for editing code. The 16k context window and functions support may have shifted that balance, so I need to dig in more.

One other thing to note is that aider now distills the repo map to fit within a token budget. The default is 1k tokens, specified via `--map-tokens`. It tries to use the budget to convey the most important parts of the repo map to GPT. So in theory, even very large repos should get a useful map.

This of course depends entirely on how big the codebase is. But the short answer is "yes".

Do you have any links / tools for this?

Here's a couple (I apologize if some of these are already obvious to you):

  - https://python.langchain.com/docs/use_cases/code/code-analysis-deeplake
  - https://twitter.com/jerryjliu0/status/1636728577524916225?s=20 (GPT repo loader)

FWIW there's tons of tools you could use to do this. GitHub Copilot is pretty close to being a comprehensive solution as well.

I wrote a simple ChatGPT plugin that exposes files within a local directory on my computer to ChatGPT [0]. It's been quite helpful for me, both for working with codebases as well as other material (supports docx and pdf as well). It does require plugin developer access to run, though.

[0] https://github.com/samrawal/chatgpt-localfiles

Why not just use ChatGPT website to chat if you dont want to handle any complexity.

Great work!

I'd love to change this to use the new function logic in chatgpt

The project has an implementation for the function logic:

https://github.com/minimaxir/simpleaichat#functions

Reply on news.ycombinator.com