Hacker News Reader

I personally think we'll have the opposite problem. I think making machines too much like humans will result in less ethical consideration of actual humans, not extra rights for machines.

Once a model becomes sufficiently like a human brain to perform many of the jobs we have in our society (driving vehicles, collecting trash, monitoring cameras and data streams, etc.) then those who have the ultimate power in our world will start to see humans as unimportant meat sacks that are inefficient compared to the machines. They'll stop pretending they care about the welfare of humanity even a little bit, and will start to push for policies that would reduce our numbers.

Eventually, I believe the end goal will be for a small number of humans to do those jobs that simply cannot be automated, and to serve an even smaller number of masters who control the machines, which will then be purposed as the police for the other humans.

That's like an interesting movie/show plot that I'd love to watch!

You write in the future tense, but I think that this is already happening. Fewer millenials are having children than previous generations, and those that are have fewer children.

I think the main driver is that most millenials simply can’t afford children, and this is a direct result of policies pushed by “those who have the ultimate power”.

Or maybe the millennials themselves see humans as unefficient meat sacks who might as well just pass the time with video games rather than striving.

Or maybe people actually think every child is precious, and those who have the ultimate power benevolently used their power to require every child get the resources it deserves, even though we can't afford it.

Ah, so the “essential workers” making minimum wage don’t deserve to have kids because they should have been striving more. I see the plan is working!

Also, if those in power require every child get the resources it deserves, doesn’t that make it easier to have kids?

Tech bros are already shamelessly talking about "artificial wombs", so... The Overton window is already open to that point

There are no actual ethical issues with contemporary ML architectures (including transformers) being too conscious or brainlike, it's just laypeople and demagogue chatter. It could be an issue in the future but only with computational systems that are utterly unlike the ones being created and used today.

Actual practitioners are tired of the debate because getting to an informed answer requires grokking some undergraduate-level math and statistics, and nobody seems particularly willing to do that

> nobody seems particularly willing to do that

This. The amount of mathematical illiteracy is staggering in ML.

The professionals I've met actually working in ML R&D have basically all been very technically competent, including in mathematics. It's more the people who talk a lot about AI in grandiose and anthropomorphized terms that I was referring to.

I work at one of the best unis in the world in a big ML research group and I have not. Unfortunately.

I even know researchers with 10k+ citations in ML that even talk about "continuous maths" and "discrete maths". This pretty much sums up their level of mathematical sophistication and ability.

What do you mean? That's an incredibly important distinction in understanding mathematics for ML in the neural net age. Perhaps a bit of a sensitive spot for me personally, coming from a harmonic analysis group for my math PhD, but the short version basically goes like: Up until the 2010s or so, a huge portion of applied math was driven by results from "continuous math": functions mostly take values in some unbounded continuous vector space, they're infinite-dimensional objects supported on some continuous subset of R^n or C^n or whatever, and we reason about signal processing by proving results about existence, uniqueness, and optimality of solutions to certain optimization problems. The infinite-dimensional function spaces provide intellectual substance and challenge to the approach, while also being limited in applicability to circumstances amenable to the many assumptions one must typically make about a signal or sensing mechanism in order for the math model to apply.

This is all well and good, but it's a heavy price to pay for what is, essentially, an abstraction. There are no infinities, computed (not computable) functions are really just finite-dimensional vectors taking values in a bounded range, any relationships between domain elements are discrete.

In this circumstance, most of the traditional heavy-duty mathematical machinery for signal processing is irrelevant -- equation solutions trivially exist (or don't) and the actual meat of the problem is efficiently computing solutions (or approximate solutions). It's still quite technical and relies on advanced math, but a different sort from what is classically the "aesthetic" higher math approach. Famously, it also means far fewer proofs! At least as apply to real-world applications. The parameter spaces are so large, the optimization landscapes so complicated, that traditional methods don't offer much insight, though people continue to work on it. So now we're just concerned with entirely different problems requiring different tools.

Without any further context, that's what I would assume your colleague was referring to, as this is a well-understood mathematical dichotomy.

I made continuous/discrete distinction more in order to take a jab at people that don't know measure theory and therefore think these approaches can't be unified. (Though I do know for the record that in some cases, like the ones you mention, there is no overarching, unifying theory.)

Other than that I agree with you with everything up to the point where you say "Without any further context...".

The dichotomy that you describe needs graduate-level mathematics to be properly understood in the first place. I'm not sure why (luck?) it seems you are biased by being surrounded by people who are competent at ML and math. I guarantee you that is not the case. If you review for any of the big conferences (e.g. NeurIPS) you will see that really fast. :(

> The professionals I've met actually working in ML R&D have basically all been very technically competent, including in mathematics.

Competent at math can mean many different things. Have they taken higher level courses in statistics, probability, optimization and control theory? If not I'd say that they aren't technically competent at math that is relevant to their field, and in my experience most don't know those things.

I don't think the undergrad math and statistics are really the problem. I think even if people understood the math, there would be some who questioned whether there's something epiphenomenon arising from the math, because there are more fundamental issues. Students in philosophy of mind classes were confronting these issues long before the DL boom started, with the Chinese Room argument. I.e. even if we accept as premise that there's a pre-described and deterministic set of steps that gives rise to seemingly intelligent behavior, we can be divided over what properties to ascribe to that system. Does it "understand"? Is it "thinking"? Is it a "person"? Why?

- we've never settled on a definition of consciousness

- we don't understand how biological brains create qualia

- we're not entirely in agreement over when human biological brains stop having ethical standing

- we disagree about how necessary embodiment or the material world outside the brain is to "mind"

- we disagree on what ethical standing to give to non-human animals with biological brains

Because we know so little, perhaps the only people that can believe strongly that current ML models are conscious have embraced panpsychism. But also because we know so little, including _what consciousness is_, I also don't think one can be confident that no ML model today has any consciousness whatsoever.

Recently I was re-reading some Douglas Hofstadter and in Metamagical Themas he has a dialogue about the "Careenium" in which on different (time)scales a physical system of balls and barriers appears to have qualitatively different kinds of dynamics. Hofstadter earnestly wants to use this to explain how "I" can direct or constrain in a top-down way some of the physical mechanisms that make me up while at the same time in a bottom-up way they comprise and animate me. I don't know if it was intentional, but it seems awfully similar to a beer-can apparatus that Searle somewhat dismissively uses as example of a potential computing substrate which could surely never support a "mind" no matter how it is programmed.

Two academics, both well respected, both who spent decades thinking about the mind and how it comes to be and how we should think about it, and both used an inscrutable complex physical (non-biological) apparatus to make points which are diametrically opposed about how the mind arises from more or less specific physical conditions.

I have in mind a property fwomp which I can't clearly define. But I have fwomp, and I'm pretty sure you have fwomp. I don't know where fwomp comes from, how it works or what are the necessary conditions for its persistence. From the preceding sentences, can you know with certainty that U-Nets don't have fwomp?

Well, you could write down a model for the full interaction between synapses and probably endow the whole thing with an RL-like environment so that you can model the interactions. The solutions of the model would then represent pretty exactly what a conscious being would do.

But it is hopeless to solve such a model, already writing it down would be a challenge.

There is some work in this direction to find a meaningful model by Eliasmith.

I think we're missing each other.

Whether you can write it down as a giant complex pile of more simple operations does not cut to the core issue: is consciousness a property of a computation which is present or not regardless of the physical substrate on which the computation happens, is consciousness a property of certain physical systems which isn't present even in extremely faithful computer simulations of those systems, or consciousness something else? (And how does ethical consideration of a system depend on consciousness?)

A computer simulation of weather doesn't have the actual wetness of rain, no matter how detailed the simulation is.

Addition can be present in a 4th-grader doing arithmetic, an abacus, an analogue computer or an electronic computer, even if they have different limitations on the size of numbers they can handle, at what speed, etc.

I'm going to ignore the 'full interaction between synapses' part b/c we don't know what kind of detail is needed for a computer model to capture the behavior of a conscious being. But the point is -- if you _had_ such a model, and could run it forward, people would still be divided as to whether it was actually conscious or not, because the basic properties of consciousness are not settled.

Ok, I see the point you wanted to make.

You ask "is consciousness a property of certain physical systems which isn't present even in extremely faithful computer simulations of those systems, or consciousness something else?" and later say "if you _had_ such a model, and could run it forward, people would still be divided as to whether it was actually conscious or not".

It seems to me that you thereby answer your first question with "no", as it seems that for you, by the last sentence, consciousness is a metaphysical property...

Which I would be fine with: I think we will only know more once we have an entity that looks and acts humanly (so there is reason to suppose consciousness might exist in it) and then try to understand its formal, inner workings and compare it to the human brain. No matter the outcome, it will be very interesting.

> It seems to me that you thereby answer your first question with "no", as it seems that for you, by the last sentence, consciousness is a metaphysical property...

That is absolutely not what I said. From my first post in this conversation I said "we've never settled on a definition of consciousness". In my preceding post, in describing a hypothetical, I said "people would still be divided as to whether it was actually conscious or not, _because the basic properties of consciousness are not settled_."

"People will disagree about X" does not in anyway amount to "I believe that X is a metaphysical property."

The absence of a clear definition means that people will continue to disagree regardless of the evidence about whichever physical system that you examine, because they're bringing different conceptions of consciousness to the table. If you know in advance that no evidence will be able to answer the question as currently framed, there's no need to wait for it; we must refine the question first.

That's not really an argument, is it? I do understand the maths behind these models, or at least a good chunk of it. I can talk about differentiable functions, softmax etc.

Regardless, going from "it involves maths" to "therefore there are no actual ethical issues" is a non-sequitur. Your conclusion doesn't follow from your premise. I actually agree with you - I don't think current transformer models like DALL-E are conscious or worthy of rights - but what you've presented here isn't something that could possibly convince anyone in either direction. You'd have to lay out why it's not and how many years of study of maths are involved really doesn't seem like a convincing basis on which to decide.

>At what point do we cross a threshold here and encounter a non-trivial number of people who argue that a sufficiently large model actually is a brain, and therefore deserves rights?

There are two ways of resolving this conundrum. You either give rights to computer programs, or take away the rights of people. Which option sounds more likely to you?

At it's heart, ethics isn't a cognitive process, but it's based on an emotional one.

We care about domesticated animals because of Oxytocin(OT), the love/connection hormone:

" Recent reports have indicated the possible contribution of OT to the formation of a social bond between domesticated mammals (dog, sheep, cattle) and humans."[1]

And sure, we'll probably create at some point an artificial creature that will release oxytocin in humans. An it's an interesting question how to design such a machine.

But most AI's ? most likely, we won't feel any strong connection with them.

[1]https://link.springer.com/article/10.1134/S2079059717030042

They're called vtubers.

Mmm, perhaps. I would be very warey of any study of the form "possible contribution of molecule X to high level social process Y", that sounds very far into the junk science danger zone.

Dogs came from wolves, originally. Nothing cute or love inducing about those. They were domesticated because they were useful and the warm feelings came later.

Sheep, cattle. We raise them for wool, milk and food. It's a purely transactional form of dominance. Very few people would profess love for a cow or a sheep outside of maybe a children's petting zoo.

Feeling creatures deserve rights, since they can suffer. Intelligence is at best indirectly related, if at all.

I think feeling is being approximated in reinforcement learning by assigning values to states, it's an important part of selecting the next action. Emotion is related to the expected reward from the current state. So an RL agent implemented with neural nets would deserve rights because it has a concept of what is good and bad vs. its goals.