omnicognate

Most of them do not have Lean proofs, and 3 of those that don't have already been found incorrect.

That's a strong contender for the worst analogy I've ever seen on HN, and it's a crowded field.

Edit: If you'd like a better medicine based one, look to radiology, where AI is an omnipresent tool but claims that radiologists are no longer needed, based on an ignorant view that a radiologist's job is "classify images according to what diseases they indicate" have only contributed to a crippling worldwide shortage of radiologists.

Guess it's no surprise hackernews has forgotten what a hacker is. Hint: Margaret Hamilton was a hacker.

Heaven forbid, as a person who is apparently "[in]capable of being impressed by anything that happens in the empirical world", that I should patronise anyone!

Good Lord yes

That's not empiricism. It's the kind of extrapolation that predicts negative Germanies and 10 ton babies. (And it's not the reason I said that, either.)

Interesting, I'll have a proper read of that. Sounds not unrelated to Kolmogorov complexity.

Can you expand on these interesting conjectures?

The fact that it's an open problem is the point I'm making. There's a very high degree of hubris right now, with people just assuming any open problem will be flattened by the AI steamroller soon. And sure, if that's what someone wants to believe that's up to them, but it's not a terribly interesting point of view to me. "What about X" "It'll be solved somehow", "What about Y" "It'll be solved somehow". Not exactly scintillating. If you know of any actual ideas on this I'd be interested to hear them.

Also any specifics on what you said about LLMs rating (preferably novel) mathematical claims for "interestingness" would be interesting.

Sure, but the question stands. The ability to evaluate some measure of success seems pretty fundamental to how we train models and iterate with them on tasks like theorem proving. What is that measure for mathematical conjecture generation? How do we evaluate success, either on a particular task for iteration (like we do by eg. setting an agent to produce a lean proof of a specific result) or on a large enough set of training data to learn a set of rules (like we do when eg. using an RL environment to train a model to generate source code that passes automated validity/correctness checks).

What's the objective function or RL environment for "interesting conjecture"? Not saying it can't be done - I no longer have any specific task that I'm confident AI won't be able to do - but I don't see how. It feels to me like something that would require a qualitatively new approach.

How?

It's a supposition that can only be proved true soon. To prove it false would take literally forever.

If I understand the abstract correctly (big caveat), they aren't saying they didn't prove it. They're saying they gave two proofs, one in natural language and one in Lean, that are not equivalent to each other. I assume the main significance is that the Lean proof is not a formal verification of the natural language one and the natural language proof is not a readable explanation of the Lean one. Both of those things can be desirable, so to complete the set we'd get 4 proofs.

He explores that, but being an entertainingly twisted sort of writer he focuses more on its use for torture, with "neural laces" in Excession and virtual hells in Surface Detail.

Did I say they don't? What I implied is that humans have other ways of thinking than trying stuff until something sticks, not that they never do that.

How do you think mathematicians came up with those open problems and conjectures - trying stuff until they came up with one that passed their "interesting conjecture" check? What was that check? Can you code it in Lean? Can ChatGPT? Have you seen an LLM come up with one?

> Its a matter of time...

No it isn't, and by saying that I've given as much reason to think I'm right as you have.

LLMs are solving outstanding maths problems by trying a load of stuff until something works, which is exactly what they do in software. If you want a system built out of square blocks jammed into round holes they'll make you one in no time, but if you're working in an area where non-functional requirements actually matter (there are lots) the slopnami is a living nightmare.

> I suspect that this is in fact the source of much of the angst.

Why do you "suspect" this as if it's some hidden motivation when the very first paragraph of the advisory group's statement (linked from the OpenAI post) says:

> At present, some frontier AI labs are testing advanced mathematical problems on proprietary models that remain inaccessible to the broader scientific community. Our recommendations are formulated with this practical context in mind. However, ideally, they would not do so. We want to state clearly from the start: we do not endorse this practice, and we ask them to stop testing advanced mathematical problems on proprietary models.

Tao and others in that group have been strongly and publicly pro AI from the start. They are not advocating "going back". They're objecting to the strip mining of open problems using proprietary technology.

That's what the suit alleges. Also that it had been automatically flagged to an internal human review team who had concluded that there was a credible, specific risk of harm to others and recommended notifying police, and that it was OpenAI leadership that rejected that recommendation.

You can see it all in GP's link. It's very readable.

"Badlands of Aram Chaos" itself sounds like a sci fi title. Also, the picture of it in the "Gallery" section is cool and visually confusing. I'm struggling not to see a smooth, reflecting water surface in it, that can't possibly be there.

Edit: I think it's an aerial picture, but my brain insists on seeing it as one taken from the surface.

I was with you up to the last sentence.

I see, that does sound like a powerful technique. Thanks!

How is this done in living brains? Presumably you have to vivisect sufficiently be able to shine light on neurons, or have some sort of probe that does it. At that point how is it easier than electrically stimulating the nerves to generate action potentials?

(To be clear, since it's probably necessary, I'm asking a curiosity-driven question about the mechanism here, not throwing shade on the work. Given it's won the researchers a Nobel prize it's obviously highly significant.)

Just needs subscription charging.

> if you wanted to moralize about enslaving your computer

That is precisely the context of the debate. You seem to find it absurd, as do I, but the reason we're talking about whether LLMs are conscious in this thread has nothing to do with whether they are useful. It's because important people, such as Dario Amodei, are engaging in exactly the kind of moralising over "AI welfare" that you're scoffing at.

You are massively over-stating the significance of the connectome. We've had the complete connectome (and lineage) for C.Elegans, with a hundredth the neurons and a 25-thousandth the synapses of a fruit fly, for decades and getting animal behaviours to emerge from simulations of it is a field very much still in its infancy.

Edit: You're also massively under-stating the significance of individual cell behaviour. I expect "[h]uman neurons are ... [v]ery similar to those neurons found in a fruit fly" would draw hollow laughter from any cellular neuroscientist. Here's a random overview of a bunch of recent papers on the importance of cellular diversity in one small part of a mammal brain: https://www.frontiersin.org/journals/cellular-neuroscience/a...

I'm not sure who the "we" in this is, but it's certainly not my position. I'm pretty sure most if not all animals are conscious. I don't think that means we shouldn't kill, eat and use them, but I do think it means industrialised animal farming as commonly practiced is a cosmic horror that is difficult to even contemplate.

I think we have an extremely strong basis on which to say AI as it currently exists does not have conscious experiences, though, and if we start trying to pretend they do some real absurdities will result. Indeed the consequences could be pretty terrible in terms of power concentration, and I doubt it's a coincidence that those best placed to form a new priesthood of AI and tell everyone how it should and should not be treated are pushing the idea so strongly.

I'm pretty sure it's possible to make conscious machines and I think it's very likely we'll eventually figure out how. The point about the spec is not "it isn't conscious because we know how it works" but "we know it isn't conscious because of what we know about how it works".

> reeks of insecurity to me

If you're going to try and debate something constructively this is not a good place to start from. I suggest avoiding making claims about the mental state and motivations of people you disagree with and instead explaining your position and asking why theirs differs. You may not agree with what they have to say but you might gain some better insights into why they think the way they do. It'll certainly be more fun than flinging insults back and forth anyway.

For my part, I'm quite certain that LLMs are not conscious despite being unable to define consciousness or specify any conclusive test for it. The reason these are not in conflict is down to the nature of consciousness and how it relates to knowledge and language.

Firstly, I know I am conscious with a higher degree of certainty than I know anything else because conscious experience is how I know anything else, up to and including that I exist (cogito ergo sum). For all I know every experience I have might be a trick. I might be a brain floating in a tank wired up to the Matrix, but even if so I am conscious.

Secondly, one of the things that I consciously experience is the relationship between consciousness itself and language in my mind. What I see is that language is smaller than, secondary to and somehow contained by consciousness. I consciously experience language but I also consciously experience much more than that, things that cannot be conveyed with language. Most of my experiences are like that, things that I could perhaps evoke in your mind by likening them to other, shared experiences, but that I could never concretely capture in language and unambiguously communicate. I know this because I experience those things and I experience language and I can perceive, directly, the categorical distinction between them.

To ask me to "define" or "prove" consciousness, then, is to ask me to capture in language the most extreme example of this categorical distinction. It's asking me to encapsulate in language not just one of the vast array of everyday experiences I have that cannot be so captured, but the entire thing, experience itself. It's just not possible.

But the fact I can't do this doesn't then imply that I don't know anything about consciousness and have no basis on which to hold a view such as "LLMs are not conscious". Indeed the reason I know I can't define consciousness is that I can perceive it and see that I can't. That same perception of and familiarity with consciousness is what enables me to form opinions about whether other people, creatures and things are likely to be conscious. I base these opinions on what I perceive of my own consciousness and what I know of how those other entities behave, came to be and work.

You can of course perfectly reasonably disagree with my opinions on those things, and you certainly have different conscious experiences than I do, but I hope the above makes it clear that the fact that I can't define or prove consciousness doesn't imply that I have no basis on which to form opinions about it.

Given the above it may surprise you to hear that I think true, conscious artificial intelligence is probably possible and that while I don't think it can be implemented by a classical algorithm I doubt it requires new physics or quantum mechanics. My speculation is that the continuum is both physically real and is (as it is represented in our existing physical theories) sufficient to allow it. Consciousness is larger than language but so is human understanding and if we keep going I expect it will eventually get us there. Nonlinear dynamics is where I'd look.

As for insecurity, I'm pretty sure spiders are conscious and that hasn't given me an inferiority complex, so I don't see why data centres being conscious would. I just don't think they are.

Give me the IEEE spec for an atom and I'll tell you.

It's a disappointingly poor article from the BBC. This one is much better and has more information: https://www.lawcommentary.com/articles/arizona-court-ai-gene...

The sentence the judge handed down was the maximum available and more than the prosecutors requested. The defence was apealing the sentencing, which was the judge's choice, not the jury's verdict. The judge was influenced by the video (as the appeals court concluded) and, regardless of the intentions of the prosecution, seeing a fake image of the victim "forgiving" the defendant could have pushed him towards a tougher sentence.

I doubt the direction of the influence actually mattered for the appeals court decision, though - they just vacated the sentence as the video wasn't allowed by law - and as the sentence was the maximum possible for the conviction it doesn't sound like there was much risk in appealing.