Claude’s memory architecture is the opposite of ChatGPT’s

The point of using an LLM is to find the thing that matches your preferences the best. As soon as the amount of money the LLM company makes plays into what's shown, the LLM is no longer aligned with the user, and no longer a good tool.

No implementation will work for very long when the incentives behind it are misaligned.

The most important part of the architecture is that the user controls it for the user's best interests.

There are 2 possible futures:

1) You are served ads based on your interactions

2) You pay a subscription fee equal to the amount they would have otherwise earned on ads

I highly doubt #2 will happen. (See: Facebook, Google, twitter, et al)

Let’s not fool ourselves. We will be monetized.

And model quality will be degraded to maximize profits when competition in the LLM space dies down.

It’s not a pretty future. I wouldn’t be surprised if right now is the peak of model quality, etc. Peak competition, everyone is trying to be the best. That won’t continue forever. Eventually everyone will pivot their priority towards monetization rather than model quality/training.

Hopefully I’m wrong.

I have never encountered such bad customer service anywhere -- and at 200 bucks a month at that.

Our goal for the router (whether you think we achieved it or not) was purely to make the experience smoother and spare people from having to manually select thinking models for tasks that benefit from extra thinking. Without the router, lots of people just defaulted to 4o and never bothered using o3. With the router, people are getting to use the more powerful thinking models more often. The router isn't perfect by any means - we're always trying to improve things - but any paid user who doesn't like it can still manually select the model they want. Our goal was always a smoother experience, not ad injection or cost optimization.

But sometimes it feels like I'm the lone voice in a bubble where people are convinced AGI is just around the corner.

I'm wondering if it's because people are susceptible to the marketing, or are just doing some type of 'wishful thinking' - as some seem genuinely interested in AGI.

LLMs will accelerate discovery and development of Innovation 1, for insanely expensive AGI.

Innovation 1 will accelerate discovery and development of Innovation 2 which will make it too cheap to meter.

- a map of the world, or concept space, or a codebase, etc

- causality

- "factoring" which breaks down systems or interactions into predictable parts

Language alone is too blurry to do any of these precisely.

Markov chains can’t deduce anything logically. I can.

https://www.anthropic.com/news/golden-gate-claude

That does actually disqualify some mechanisms from counting as intelligent, as the behaviour cannot reach that threshold.

We might change the definition - science adapts to the evidence, but right now there are major hurdles to overcome before such mechanisms can be considered intelligent.

It has no intelligence. Intelligence implies thinking and it isn’t doing that. It’s not notifying you at 3am to say “oh hey, remember that thing we were talking about. I think I have a better solution!”

No. It isn’t thinking. It doesn’t understand.

whats the benefit of calling something "intelligent" ?

Current LLMs are a long way from there.

You may think "sure seems like it passes the Turing test to me!" but they all fail if you carry on a conversation long enough. AIs need some equivalent of neuroplasticity and as of yet they do not have it.

The idea that "understanding" may be able to be modeled with general purpose transformers and the connections between words doesn't sound absolutely insane to me.

But I have no clue. I'm a passenger on this ride.

I don’t know if AGI needs to have all human traits but I think a Markov chain that sits dormant and does not possess curiosity about itself and the world around itself does not seem like AGI.

I don’t think humans are the only ones to have both these things but that’s what I think of as a way to divide species.

(Among these are “preserve your ability to further your current goals”)

The usual analogy people give is between natural selection and the gradient descent training process.

If the training process (evolution) ends up bringing things to “agent that works to achieve/optimize-for some goals”, then there’s the question of how well the goals of the optimizer (the training process / natural selection) get translated into goals of the inner optimizer/ agent .

Now, I’m a creationist, so this argument shouldn’t be as convincing to me, but the argument says that, “just as the goals humans pursue don’t always align with natural selection’s goal of 'maximize inclusive fitness of your genes' , the goals the trained agent pursues needn’t entirely align with the goal of the gradient descent optimizer of 'do well on this training task' (and in particular, that training task may be 'obey human instructions/values' ) “.

But, in any case, I don’t think it makes sense to assume that the only reason something would not obey is because in the process that produced it, obeying sometimes caused harm. I don’t think it makes sense to assume that obedience is the default. (After all, in the garden of Eden, what past problems did obedience cause that led Adam and Eve to eat the fruit of the tree of knowledge of good and evil?)

- With the new GPT Voice, I have basic, planned conversations. Let's go to a restaurant. Let's say we're friends who ran into each other. etc...

- I use it for quizzes. "Let's work on these verbs in these tenses. Come up with a quiz randomly selecting a verb and a tense and ask me to say real world sentences." "Quiz me on the numbers one through twenty".

- I am using it to help learn the Hindi script. I ask it to write childrens stories for me, but I ask it to write each line in the hindi script, then phonetic spelling of the hindi script, and then in english so I can scroll down and see only the hindi first, then if I have issues I can see the phonetic spelling of the hindi. Then I can try to translate it and then check the english translation on the third line.

Those are the main things I'm doing. I don't know if I'll ever be fluent, but I find if you work on these basic ever day conversations you can have a conversation with someone. If you speak a language for the first time around a native speaker it's usually very predictable. They'll ask how long you've been learning, where did you learn, have you been to <country>, and you can direct the conversation by saying things about where you live and your family, etc... That's the base I'm building and it's fun. If you're not doing at least 30 minutes a day you're never going to learn a language, you probably need an hour more a day to really get fluent.

LLMs are understood to the extent that they can be built from the ground up. Literally every single aspect of their operation is understood so thoroughly that we can capture it in code.

If you achieved an understanding of how the human brain works at that level of detail, completeness and certainty, a Nobel prize wouldn't be anywhere near enough. They'd have to invent some sort of Giganobel prize and erect a giant golden statue of you in every neuroscience department in the world.

But if you feel happier treating LLMs as fairy magic, I've better things to do than argue.