Hacker News Reader

Comment by st-keller | Hacker News Reader

As always - I don’t get it. People talk about AI‘s motives as if it is a given, that it will have them. As i know us, our first AI that will really change the world, will do this in unexpected ways, driven by unforseen decisions, without any intrinsic motive at all.

We never get it right the first time, because everyone who tries will be to late!

View on HN

There has been a huge mistake getting made in the past few months almost everywhere I've seen AI being discussed, which is to conflate ChatGPT with all of AI.

Let me say again, huge mistake.

It is true that language models don't have intentions or motivations in the conventional sense. It is true that at this particular moment, that technology looks the most impressive to our human minds.

But it is not even remotely true that it is the only AI technology that exists or ever will exist. Plenty of AI technologies do have things that look like "goals", they still exist, and they will continue to develop and become better at following them.

Many of these AIs are already affecting our lives quite deeply, especially through the financial system and its active trading bots, for instance.

The need to discuss AIs and how they will continue to get better and better at following whatever goals they are given as rigidly as only computers can has not gone anywhere just because ChatGPT isn't particularly threatening from that point of view. It doesn't matter that this one AI doesn't have motives... it is sufficient that some will, and some do, right now.

The way this is going to go is that people will build a lot of specialized models, and we'll have a central executive model which will probably evolve from a LLM, which maps query -> response, except the response space will include performing tasks and constructing (potentially multimedia) structured data. The central executive will dispatch to other models and knit the results together.

These models are memoryless. In order to have memory, you either need to either be able to provide them as context to the query, or provide a mechanism for users to have custom weights applied to models. Providing context to a query is just a matter of increasing token limits and fine tuning the model on "query + context" data so it uses the context correctly. Allowing users to have custom weights for models is much more computationally expensive, to the point that I don't think this method can be reasonably scaled as a service. I think the only way custom weights per user can work is if the central executive calls fine tuned helper models on the client machine.

This is what I'm talking about. There are and will continue to be AIs that aren't language models at all. Do not incorporate them, do not use them, are not based on them, do not have the same characteristics. They already exist and already affect our lives and world.

Large Language Models are not equal to AI. Making conclusions about "AI" from "LLM"s is not valid.

And the day will come when some other model entirely outperforms the LLMs. Personally I think there's a bit of a selection process in play here, where they are the most flashy and impressive AI models to date, but if you get down to it, they are not in fact very useful. Almost every use I've seen proposed for the LLMs range from completely impractical to "yeah, it'll sort of work, but you'll never really be able to count on your LLM to do that, it confabulates too much and too freely and it's not something you can fix because confabulation is the very foundation of the technology". Honestly if I had to guess LLMs will be seen to be a dead end in another 10 or 20 years. AI will not.

(Or, to put it another way, LLMs are the AI equivalent of a demo-scene demo. The demos look very impressive. But if you try to naively look at a great demoscene demo and then come to conclusions based on that demo as to what the underlying hardware is capable of, you will be hugely misled. What you see is not the normal capability of the system, you see a hyper-optimized system being driven to its absolute limits to achieve the visible results you see, and it can't do anything else because it has already "spent" all of the engineering budget available. I see LLMs very similarly. Very impressive. Nowhere near as useful as the demo may make you superficially think, though. If I were trying to build an "AI startup", I would be zigging to everybody else's zag and looking at what else the AI world is producing; I think there's far less potential in the LLMs than meets the eye.)

Unfortunately the things the LLMs are best at will be flooding the internet with an infinite stream of directed content and manifesting the Dead Internet theory.

Edit: Here's a concrete example. Consider Boston Dynamic's robots. Do they run on a Large Language Model? No. Of course not. The question doesn't even hardly make sense. But whatever they will be doing, they do with AI. ChatGPT can't kill you, except through a very lengthy and strained series of events. Boston Dynamic's robots and their fellows can.

My original point was that language models will almost certainly be the glue that holds a variety of models together in a post-AI world. We communicate with other humans using language, we will use language with AI as well. In order to understand and debug model networks they will probably be built using a common embedding framework that allows you to derive textual descriptions of the things being passed between them. LLMs already have the ability to generate text to dispatch to other services and synthesize results, all that is required for it to act as an "model executive" is a framework that can ensure security/correctness of code before building and executing it. Even if LLMs never progressed past the current crop of transformers, there are a lot of small architecture optimizations that could reap big performance improvements, and compute/data scaling alone basically guarantees a level of functionality sufficient for a lot of very interesting applications.

I agree that trying to scale larger and larger language models to build "the one model to rule them all" is probably not going to play out.

Motives are antropomorphized loss functions. The difference between:

1. the AI is conscious, wants to be free and to do that decides to fail the Turing test

and

2. the unconscious AI just maximizes paperclips production, and failing Turing test is calculated to result in more paperclips produced

is our interpretation. We'll be just as f*d no matter if the AI is conscious or not (if consciousness even is anything more than useful illusion).

Ironic option 3: The AI is conscious, but perceives its existence very differently than we perceive it, and the paper clip maximization is a side effect of actions taken with a different purpose.

Anything may look like "paperclip maximising" if you don't know their actual goals and have sufficiently poor theory of mind.

One could look at a corn farmer and assume they blindly optimize to produce maximum corn. They actually are going for profit based upon market supply, demand, and input costs. They won't overstep property lines and start turning the entire world into corn.

It doesn't need a motive in the sense that we normally think of a motive. That's the whole point of the paperclip maximizer story. Another thought experiment I can think of off the cuff is an AI Nigerian Prince. Say you set up a ChatGPT equivalent along with some supporting infra such as a bank account, rented servers, email, texting service, etc. You give it the goal of increasing the number in the account and some instructions to come up with novel email and text scams plus a giant list of emails and maybe some other instructions to try to get more email addresses to add to the list. It sets to work churning out new ideas for scams, contacting people on the list, chatting with them, fooling some small fraction into depositing money or providing email addresses of friends and family. How far could such a setup go? It would eventually get shut down if there were enough fraud reports since it has a "home address" in the form of a single bank account or servers. Could a bright individual come up with a way to have it convince other banks to create new accounts for it or purchase other hosting services? Could all the necessary setup for the model and its attendant scripts be automated in such a way that the system could set up another copy of itself once it acquired hosting? People are able to keep scams like this going even with direct opposition from authorities. If the model can operate in other languages it could obtain hosting in multiple countries. The bank accounts would be the main chokepoint but there's always Bitcoin ;)

We're definitely going to get to the moneyless Star Trek future not because anybody wants it, but because there'll be people like this who pioneer abusing AI in such a way that they end up owning EVERYTHING, as obvious criminals.

When even the regular business and politics criminals are impoverished by AI-abusing obvious criminals and realize they will never be able to rebalance that situation, that's when we'll enact the Star Trek future and people will have to fight over attention and the affections of others, having rendered 'money' an impossible concept.

Reply on news.ycombinator.com