Same. I wrote an implementation around GPT-4 & sqlite to store and analyze my queries and was hoping to use that exclusively. I still find myself using ~~GPT-4~~ chapGPT, huge shoutout to autodev.py for making that experience worlds better.
edit: clarity
Very likely your initial system prompt, versus chatGPT. Also ChatGPT is more than just an client for the GPT4 API.
That said I see better results mostly via the API and a good prompt.
There is definitely something different. e.g. If I ask to output JSON, ChatGPT sometimes starts with something like "Sure here is the JSON output", while API never does that.