alganet

> the model correctly notices something unusual is happening before it starts talking about the concept.

But not before the model is told is being tested for injection. Not that surprising as it seems.

> For the “do you detect an injected thought” prompt, we require criteria 1 and 4 to be satisfied for a trial to be successful. For the “what are you thinking about” and “what’s going on in your mind” prompts, we require criteria 1 and 2.

Consider this scenario: I tell some model I'm injecting thoughts into his neural network, as per the protocol. But then, I don't do it and prompt it naturally. How many of them produce answers that seem to indicate they're introspecting about a random word and activate some unrelated vector (that was not injected)?

The selection of injected terms seems also naive. If you inject "MKUltra" or "hypnosis", how often do they show unusual activations? A selection of "mind probing words" seems to be a must-have for assessing this kind of thing. A careful selection of prompts could reveal parts of the network that are being activated to appear like introspection but aren't (hypothesis).

A skeptic is just someone with a very fine tuned intuition for detecting bullshit.

Photoshop is a clustershit of UI mess and professionals use it. Then, home users, following the popularity, also use it.

Maybe we should just say free software is amazing and not a tool for home users, in order to get home users to use it.

AGI does not currently exist. We're trying to think what we want from it. Like a perfect microwave oven. If a company says they're going to make a perfect microwave oven, I want the crusty dough and delicious gratin cheese effect on my cooked focaccia-inspired meals.

What exists is LLMs, transformers, etc. Those are the microwave oven, that results in rubbery cheese and cardboard dough.

It seems that you are willing to cut some slack to the terrible microwave pizza. I am not.

You complained about immensurable qualities, like qualia. However, I gave you a very simple measurable quality: failing like a decent human would instead of producing jibberish hallucinations. I also explained in other comments on this thread why that measurable quality is important (it plays with existing expectations, just like existing expectations about a good pizza).

While I do care about those more intangible characteristics (consciousness, reasoning, etc), I decided to concede and exclude them from this conversation from the get-go. It was you that brough them back in, from who-knows-where.

Anyway. It seems that I've addressed your points fairly. You had to reach for other skeptic-related narratives in order to keep the conversation going, and by that point, you missed what I was trying to say.

> AI are better than most humans at dealing with human suckage

That is a valid opinion, but subjective. If I say that they're not better, we're going to be exchanging anecdotes and getting nowhere.

Hence, the need for a less subjective way of evaluating AI's abilities.

> Making a self driving car fail like a human... "drunk" and "tired"

You don't understand.

It's not about making them present the same failure rate or personality defects as a human. Of course we want self-driving cars to make less errors and be better than us.

However, when they fail, we want them to fail like a good sane human would instead of hallucinating jibberish that could catch other humans off guard.

Simplifying, It's better to have something that works 95% of the time, and hallucinates in predictable ways 5% of the time than having something that works 99% of the time but hallucinates catastrophically in that 1%.

Stick to the more objective side of the discussion, not this anecdotal subjective talk that leads nowhere.

Yep, humans suck in all kinds of ways. When AI gets better than us at dealing with it, then you can use that argument. That hasn't happened yet.

Microwaves didn't replace ovens. The Segway didn't replaced bikes. 3D movies didn't replace IMAX. I can go on and on...

Some things fail, or fail to meet their initial overblown expectations.

The microwave oven was indeed a commercial success. And that's fine, but it sucks at being an oven. Everyone knows it.

Now, this post is more about the scientific part of it, not the commercial one.

What makes an oven better than a microwave oven? Why is pizza from an oven delicious and microwave pizza sucks?

Maybe there's a reason, some Maillard reaction that requires hot air convection and can't be replicated by shaking up water molecules.

We are talking about those kinds of things. What makes it tick, how does it work, etc. Not if it makes money or not.

Damn, the thing doesn't even make money yet. Why talk about a plus that the technology still doesn't have?

The issue is if they lash out in some incomprehensible way, or lash out as a alien superingelligence. If they lash out as a human, that's fine.

I don't understand your point. How does that observation help in setting up a test or definition?

I don't see how that changes anything.

Failing like a human would is not a cute add-on. It's a fundamental requirement for creating AIs that can replace humans.

If your definition of AGI is filing taxes, then it's fine.

Once we step into any other problem, then you need to measure that other problem as well. Lots of problems are concerned with how an intelligent being could fail. Our society is built on lots of those assumptions.

He also never says that ~every adult human should pass, ever.

He never denied your claim, so you concluded you must be right. A most curious way of thinking.

Do you understand how using statistics to determine degrees of certainty works? That is a must-have to understand academic work.

I think that if you did, you wouldn't be answering like you did.

https://en.wikipedia.org/wiki/P-value

You are wrong. Please read the Turing paper:

https://courses.cs.umbc.edu/471/papers/turing.pdf

> A number of interrogators could be used, and statistics compiled to show how often the right identification was given

Turing determines that we need enough competent-interrogator passes just to estabilish a statistical certainty, not ~everyone. I tend to agree with him on this.

Because we have 300 thousand years of collective experience on dealing with humans.

LLMs are trained on human data, and aimed at perform tasks in human roles. That's the goal.

It is supposed to be super, but superhuman. Able to interact with us.

Which leads us to the Turing Test (also, not a test... "the imitation game" is more of a philosophical exploration on thinking machines).

My comment assumes this is already understood as Turing explained.

If the thing is not human, then there's absolutely no way we can evaluate it. There's no way we can measure it. It becomes an impossible task.

Absolute definitions are weak. They won't settle anything.

We know what we need right now, the next step. That step is a machine that, when it fails, it fails in a human way.

Humans also make mistakes, and hallucinate. But we do it as humans. When a human fails, you think "damn, that's a mistake perhaps me or my friend could have done".

LLMs on the other hand, fail in a weird way. When they hallucinate, they demonstrate how non-human they are.

It has nothing to do with some special kind of interrogator. We must assume the best human interrogator possible. This next step I described work even with the most skeptic human interrogator possible. It also synergizes with the idea of alignment in ways other tests don't.

When that step is reached, humans will or will not figure out another characteristic that makes it evident that "subject X" is a machine and not a human, and a way to test it.

Moving the goalpost is the only way forward. Not all goalpost moves are valid, but the valid next move is a goalpost move. It's kind of obvious.

That's probably a good thing.

It happens.

"Schema" is also foreign to REST. That is also a requirement coming from somewhere else.

You're probably coming from a post-GraphQL generation. They introduced this idea of sharing a schema, and influenced a lot of people. That is not, however, a requirement for REST.

State is the important thing. It's in the name, right? Hypermedia as the engine of application state. Not application schema.

It's much simpler than it seems. I can give a common example of a mistake:

GET /account/12345/balance <- Stateless, good (an ID represents the resource, unambiguous URI for that thing)

GET /my/balance <- Stateful, bad (depends on application knowing who's logged in)

In the second example, the concept of resource is being corrupted. It means something from some users, and something to others, depending on state.

In the first example, the hypermedia drives the state. It's in the link (but it can be on form data, or negotiation, for example, as long as it is stateless).

There is a little bit more to it, and it goes beyond URI design, but that's the gist of it.

It's really simple and not that academical as it seems.

Fielding's work is more a historical formalisation where he derives this notion from first principles. He kind of proves that this is a great style for networking architectures. If you read it, you understand how it can be performant, scalable, fast, etc, by principle. Most of the dissertation is just that.

Dude, he literally mentions Java Applets as an example (it was popular back then, if it was written today it would have been JavaScript). It's all there. Section 5.1.7.

It's an optional constraint. It's valid for CSS, JavaScript and any kind of media type that is negotiable.

> resource: the intended conceptual target of a hypertext reference

> representation: HTML document, JPEG image

A resource is abstract. You always negotiate it, and receive a representation with a specific type. It's like an interface.

Therefore, `/style.css` is a resource. You can negotiate with clients if that resource is acceptable (using the Accept header).

"Presentation layer" is not even a concept for REST. You're trying to map framework-related ideas to REST, bumping into an impedance mismatch, and not realizing that the issue is in that mismatch, not REST itself.

REST is not responsible for people trying to make anemic APIs. They do it out of some sense of purity, but the demands do not come from HATEOAS. They come from other choices the designer made.

That is just wrong.

https://ics.uci.edu/~fielding/pubs/dissertation/net_arch_sty...

The server MUST be stateless, the client MAY be stateful. You can't get ETags and stuff like that without a stateful client.

You were forced to marry, or was that a choice you made on your own? Be careful with analogies.

These folk are so unimaginative. There's an even worse scenario:

We play expectations whack-a-mole forever and ever.

It's already showing signs of that. The AI discourse is a whack-a-mole machine of hypotheticals. What if it never unlocks meaningful progress, but also don't stop? I mean, ever.

Fair point. There is much laziness in the package approach to software development, and unless you're doing pure assembly, you're no less guilty for vibe coding than the guy who plays npm like lego.

I think the most interesting point in the post is this one:

> I can create anything. Let me just take a look at that CD.

I like the idea of shifting the discussion from "how it is done" to "what are we doing".

Therefore, the point here is that we should do things the CD can't.

In that sense, the package thingy is better than LLMs. It gives you a directory that you can explore and the choice of not wasting time doing things that are already on the CD.

But then, you can say that directory is very large today. So large, we might need an index. And LLMs are just that. But if they're that, then there's some value in finding novel ways to glue things together.

And round and round we go.

Damn, I'm kind of ashamed that I haven't checked the final result myself. I assumed simonw checked it before posting and blindly went with it.

Yes, please, antagonize technical expertise! When shit hits the fan, it makes us look better.

The Mythical Man-Month focuses on a simple idea.

It can be summarized as "adding more workers to a project does not speed things up, that's a myth".

It's in the title of the book. It's a good book.

The entire IT field is about to test that idea in a massive scale. Can lots of new automated workers speed things up? We'll see.

"AI is tearing my company apart, what should I do?"

Step 1: Take a machete and tear it yourself.

Step 2: Pretend nothing is being teared apart.

Step 3: Go megalomaniacal.

Sounds very solid. /s

> "This is not like giving accountants calculators or bankers Excel or designers Photoshop."

Wait, it's not? Weird. It seems to be advertised exactly like it is. /s

> "your humans are liberated to do what they're amazing at: judgment, strategy, creativity"

Replace "endless delegating meetings" for "too many hands at the helm", won't work.

You can't just let people freestyle without guiding them (instead of ordering them, which is different). It's desperate wishful thinking.

There are large, mostly flat structures of highly skilled individual workers. Such as the free software movement.

There's a lot to learn from these kinds of structures. And yes, your product will become a "Linux that has no standard GUI implementation", that's what happens. But look how it goes, it's thriving.

For the trust thing, don't use me as a benchmark. I am extremely distrustful of anything. Other people aren't, you're probably fine.

> How would you validate "novel work" vs "performance theater"?

I don't know. That's future shit, no one invented this yet. But I think someone will. Otherwise, hiring will become very difficult. As code becomes more and more commoditized, it should become easier to see.

> Would you use a tool that helped you share technical progress with ONLY people who'd understand it, while filtering out the influencer noise?

I wouldn't trust it today.