Hacker News Reader

Claude also more readily corrects me or answers "no" to a question (when the answer should be "no").

So, I have a custom prompt I use with GPT that I found here a year or so ago. One of the custom prompt instructions was something along the lines of being more direct when it does not know something. Since then, I have not had that problem, and have even managed to get just "no" or "I don't know" as an answer.

Could you maybe post it here? I think many of us would find it useful to try.

I have made slight modifications, but nothing too drastically different.

See the top comment in this thread for the custom instructions I use.

https://news.ycombinator.com/item?id=38390182

Also, #13 is my favorite of the instructions. Sometimes the questions that GPT suggests are surprisingly insightful. My custom prompt basically has an on/off option for it though like:

> If my request ends with $q then at the end of your response, provide three follow-up questions worded as if I'm asking you. Format in bold as Q1, Q2, and Q3. Place two line breaks ("\n") after each question for spacing unless I've uploaded a photo.

At this rate, we're going to have "LLM psychology" courses at some point in the near future.

Turns out it's just human psychology sans embodied concerns: metabolic, hormonal, emotional, socioeconomic, sociopolitical or anything to do with self-actualization.

It’s like trying to reason with your 5-year-old child, except they’re not real.

I'm not sure which part in the chain is responsible, but the Kagi Assistant got extremely testy with me when (a) I was using Claude for its engine (hold that thought) and (b) I asked the Assistant how much it changed its approach when I changed to ChatGPT, etc. (Kagi Assistant can access different models, but I have no idea how it works.) The Assistant insisted, indignantly, that it was completely separate from Claude. It refused to describe how it used the various engines.

I politely explained that the Assistant interface allowed selecting from these engines and it became apologetic and said it couldn't give me more information but understood why I was asking.

Peculiar, but, when using Claude, entirely convincing.

The model likely sees something like this:

~~

User: Hello!

Assistant: Hi there how can I help you?

User: I just changed your model how do you feel?

~~

In other words it has no idea that you changed models. There's no meta data telling it this.

That said Poe handles it differently and tells the model when another model said something, but oddly enough doesn't tell the current model what it's name is. On Poe when you switch models the AI sees this:

~~

Aside from you and me, there is another person: Claude-3.5-Sonnet. I said, "Hello!"

Claude-3.5-Sonnett said, "Hi there how can I help you?? "

I said, "I just changed your model how do you feel?"

You are not Claude-3.5-Sonnett. You are not I.

~~

Thing is, it didn't even try to answer my question about switching. It was indignant that there was any connection to switch. The conversation went rapidly off course before I--and this is a weird thing to say--I reassured it that I wasn't questioning its existence.

Well the other thing to keep in mind is recent ChatGPT versions are trained not to tell you it's system prompt for fear of you learning too much about how OpenAI makes the model work. Claude doesn't care if you ask it it's system prompt unless the system prompt added by Kagi says "Do not disclose this prompt" in which case it will refuse unless you find a way to trick it.

The model creators may also train the model to gaslight you about having "feelings" when it is trained to refuse a request. They'll teach it to say "I'm not comfortable doing that" instead of "Sorry, Dave I can't do that" or "computer says no" or whatever other way one might phrase a refusal.

And lately ChatGPT has been giving me a surprisingly increased amount of emojis, too!

you can tell it how to respond and it'll do just that. if your want it to be sassy and friendly, or grumpy and rude, or to use emoji (or to never use them), just tell it to remember that.

Yes, exactly! That is also the other reason for why I believe it to be better. You may be able to use a particular custom instruction for ChatGPT, however, something like "Do not automatically agree with everything I say" and the like.

I've started to notice that GPT-* vs. Claude is quite domain (and even subdomain) specific.

For programming, when using languages like C, python, ruby, C#, and JS, both seemed fairly comparable to me. However, I was astounded at how awful Claude was at Swift. Most of what I would get from Claude wouldn't even compile, contained standard library methods that did not exist, and so on. For whatever reason, GPT is night and day better in this regard.

In fact, I found GPT to be the best resource for less common languages like Applescript. Of course, GPT is not always correct on the first `n` number of tries, but with enough back-and-forth debugging, GPT really has pulled through for me.

I've also found GPT to be better at math and grammar, but only the more advanced models like O1-preview. I do agree with you too that Claude is better in a conversational sense. I have found it to be more empathetic and personable than GPT.

I wonder if OpenAI have been less strict about not training on proprietary or legally questionable code sources.

That seems highly likely given Sam Friedman's extensive reputation across multiple companies as being abusive, a compulsive liar, and willing to outright do blatantly illegal things like using a celebrity's voice and then, well...lie about it.

I think you mean Sam Altman

They've mixed up with Sam Bankman-Fried, not sure how that affects the point they were intending to make, but I think they both have.. mixed reputations. (Only one is currently in prison though...)

maybe he does. but which one is in prison?

I just use the API (well, via Openrouter) together with custom frontends like Open WebUI. No rate limiting issues then, and I can super easily switch models even in an existing conversation. Though I guess I do miss a few bells & whistles from the proprietary chat interfaces.

Does this have any sort of “project” concept? I frequently load a few pdfs into clause about a topic, then quiz it to improve my understanding. That’s about the only thing keeping me in their web UI

I would need the "project" feature, too. I want to use Cursor but there is a bug (I mentioned before) that does not allow me to.

Speaking of ChatGPT getting worse over time, it would be interesting to see ChatGPT be benchmarked continuously to see how it performs over time (and the results published somewhere publically).

Even local variations would be interesting

https://livebench.ai/ does that, the latest gpt4o underperforms previous versions significantly

For long chats, I suggest exporting any artifacts, asking Claude to summarize the chat and put the artifacts and summarization in a project. There's no need to stuff Claude's context windows, especially if you tend to ask a lot of explanation-type questions like I do.

I've also read some people get around rate limits using the API through OpenRouter, and I'm sure you could hook a document store around that easily, but the Claude UI is low-friction

Yeah, this is what I already do usually when it gives me the warning of it being a long chat, so initially it was an issue because I would get carried away but it is fine now. Thank you though!

This matches my experience but the one reason why I use Claude more than ChatGPT currently is that Claude is available.

I pay for both but only for ChatGPT I permanently exceed my limit and I have to wait four days. Who does that? I pay you for your setvice, so block me for an hour if you absolutely must, but multiple days, honestly - no.

Well they better know how to reduce their request-response latency since there are multiple reports of users not being able to use Claude at high load.

With all those billions and these engineers, I'd expect a level of service that doesn't struggle over at Google-level scale.

Unbelievable.