Artificial Intelligence – The Revolution Hasn’t Happened Yet (2018)
medium.com
medium.com
I'm not a data scientist and I've never encountered that term "provenance" before but I've encountered the problem he talks about in the wild here and there and have searched for a good way to describe it. His ultrasound example is a great, chilling, example of it.
I also like the term "Intelligence Augmentation" (IA). I've worked for a couple companies who liberally sprinkled the term AI in their marketing content. I always rolled my eyes when I came across it or it came up in say a job interview. What we were really doing, more practically and valuably, was this: IA through II (Intelligent Infrastructure), where the Intelligent Infrastructure was little more than a web view on a database that was previously obscured or somewhat arbitrarily constrained to one or two users.
https://medium.com/@rreisman/as-we-will-think-the-legacy-of-...
https://en.wikipedia.org/wiki/Provenance gives more on the term and the way it shows up.
More interestingly, both wine and art have had their provenance hinges widely exploited for massive profit while posh people think they're enjoying something exclusive.
course, I've also never heard of or touched the software you listed there either, but that may be because I don't view the data science and machine learning I'm interested in as being about specific software or vendor software...
sounds more database- lingo to me...
"Provenance" just means where the data came from. [1]
It's one of those shibboleths and terms of art used by people in industry. If you go to trade-shows you'll hear it being used -- it's worth knowing if nothing else but for its sociological value among the data software tools crowd.
Side: it's a little like the word "inference" being used as a verb by folks in AI (example usage: we use GPUs to speed up model "inferencing") -- in AI, to inference means to "predict". It's a term of art. If someone with a traditional statistics background went to a deep learning conference, they are likely to be very confused because in traditional statistics, inference means to obtain parameters θ in a model y = f(x,θ), whereas in AI, inferencing refers to obtaining y.
[1] https://en.wikipedia.org/wiki/Data_lineage#Data_provenance
Where I work, (a large pharmaceutical), these notebooks are taken very seriously by biologists, chemists, and chemical engineers, and increasingly are shaping the mindset of our data scientists (who have yet to adopt them).
Given the longstanding practice of documenting experiment design and method, I think it's probably long overdue that the exploratory analysis of experiment-based data must also adopt more rigorous governance to ensure that necessity and sufficiency are ensured when drawing inferences from experiment, especially when the data used was not originally intended to answer the current question posed.
A few publications from ~2011-2015 period:
http://ceur-ws.org/Vol-1558/paper37.pdf
https://ieeexplore.ieee.org/document/5739644
https://link.springer.com/chapter/10.1007/978-3-642-53974-9_...
Add a variety of additional links dating back a bit further (note the emphasis in this case on research data and tracking state of an experiment).
https://nnlm.gov/data/thesaurus/data-provenance
Data provenance is not a database / data warehouse term. It is uniquely and specifically a basic “101” concept of statistical science and ML / data science, where the custody and tracking of data are specifically tied to iterations of experiments, prototypes and research, for the sake of reproducibility.
If I was interviewing an experienced statistical researcher and they didn’t at least have a working knowledge of the core concepts, that would be a huge red flag.
Another poster mentioned vendor brochures and trade shows, which is in line with my expectations about which community it stems from, and also explains why I've never heard of it because I try to keep away from such environments these days.
Everywhere I've been the things which I take to make up "provenance" have generally been referred to under the simple label of "data quality", with separate subset definitions and measures such as timeliness, source, authority, format, history, suitability, verification, etc.
Of course, that's assuming people even worry about such things. In practice, let's be frank, anyone who's worked with data science knows they actually get shorter shrift than they deserve in practice: I'm probably among a minority of people in the real world who actually take things seriously, and I find myself on a constant crusade to remind people that just because a data point exists in a data set doesn't mean it's useful/ appropriate/ truthful/ unbiased.
data quality is a bit problematic, because I can see it being used by people who think provenance doesn't have any thing to do with quality, and from a variety of fields, but it is also infinitely more popular according to historical search trends, and in my last three jobs provenance would fall under the data quality framework.
In all of them, data provenance has been a first class consideration of machine learning and data platform teams, like a day-to-day concern and baked in to architecture review guidelines and production checklists and whatnot for every ML project.
In many of these companies we had teams of 20-40 ML scientists, all of whom knew about data provenance as a first class consideration in their work, had experience with it from their past jobs and academic programs, and considered it on equal footing with any aspect of data curation, model selection, model training and model serving.
Shrug, such is life I guess. That being said, I care deeply about this stuff (but didn't have a word), so perhaps it will be easier to convince people to pay attention to the data with said word.
I’ve worked as a data engineer for the last two years and never heard of this being used in this context before.
Typically the word “data lineage” is used to mean this in my experience.
I don’t think I’ve ever been in a meeting where someone mentioned provenance except referring to a show about paintings.
Lineage isnt the same thing, being a more specific technical term referring to keeping the history of datasets and where they came from (basically), but people actually say the words “data governance” and “lineage”.
edit: I should add that I'm definitely in favour of having provenance in ML systems, and libraries layered on top are the way that people currently do that. It's just odd that people aren't working on adding that support directly into scikit-learn/TF/pytorch etc.
I totally agree that where you have framework hooks, you should have provenance, but given there's no standard for what provenance is, no defacto open-source platform, the sklearn and tf and pytorch folks rightly steer clear. We see that if you have a shared file system, you can use conventions for path names (features go in 'featurestore', training data in 'training', models in 'models', etc), to capture a ton of provenance data.
Professor Margo Seltzer (https://www.seltzer.com/margo/) is a well-known researcher in the area of provenance. I highly recommend reading her papers if you're interested, starting with her USENIX ATC 2006 paper "Provenance-Aware Storage Systems".
But the money is in replacing humans.
We had built it precisely to free us from certain repetitive things in machine learning projects [environment set up, near real-time collaboration on notebooks, scheduling long-running notebooks, experiment tracking, model deployment and monitoring]. We used to scramble and do all that, request help from our colleagues and pull them from what they were doing. This was really taxing and bad for morale, jumping around from one context to another.
I had a huge smile contemplating all the work I was about to not do.
There are many things where the humans themselves ought to be "augmented". Case in point, in some projects involving predictive maintenance, the stakes of an incident can be around $100MM and all these processes depend on a human being alert at all times during their very long shift, with a bunch of other things happening simultaneously. This is very stressful and these people actually want to be "augmented". They want something to help them and catch things they would have missed because they haven't had proper sleep or were too busy solving another urgent and important problem. It is the people themselves who come to us and ask us for our help to help them solve these problems.
It may sound cheeky, and in many cases at many companies it is cheeky and it is PR like saying "partners" instead of "drivers", or "dashers" instead of "delivery person". In some cases it really is what happens. At least from my biased perspective with the actual humans who were asking for "augmentation" to do their job.
Any time a tool evolves, it changes not only how a task is done but also why. In the case of implementing a business process, the revision process is best served by reconsidering why what is done now and taking the opportunity to evolve the old role into making a richer contribution that introduces a new and improved path through the problem space.
That's IA, and IMO, it's the Great White Hope that AI might yet lead to a future world that engages humans more rather than the default dystopia where we're all redundant and irrelevant.
That URL doesn't seem to be the original source though.
"Adaptive Intelligence" might be described as the ability to be given a few instructions, gather some information and take actions that accomplish the instructions. It's what "underlings", "minions" do.
But if we look at deep learning, it's almost the opposite of this. Deep learning begins with an existing stream of data, a huge stream, large enough that the system can just extrapolate what's in the data, include data leads to what judgements. And that works for categorization and decision making the duplicates what decisions humans make or even duplicates what works, what wins in a complex interaction process. But all that doesn't involve any amount of adaptive intelligence. It "generalizes" something but our data scientists have no idea exactly what.
The article proposes an "engineering" paradigm as an alternative to the present "intelligence" paradigm. That seems more sensible, yes. But I'm doubtful this could accepted. Neural network AI seems like a supplement to the ideology of unlimited data collection. If you put a limit on what "AI" should do, you'll put a limit on the benefits of "big data".
You can also put any algorithm you want inside a neural net as long as you have a mechanism to pass gradients back - for example in the final layer you could have a complex graph-matching algorithm to map the predictions to the target, or you could put an ODE solver as a layer, or a logic engine, or a database.
The AI revolution is very likely something that will require a fundamental reset of our understanding of the problem domain. We need to identify a way to attack the problem in such a way that we can incrementally scale all aspects of intelligence.
The only paradigm that I am aware of which seems to hint parts of the incremental intelligence concept would be the relational calculus (aka SQL). If you think very abstractly about what a relational modeling paradigm accomplishes, it might be able to provide the foundation for a very powerful artificial intelligence. Assuming your domain data is perfectly normalized, SQL is capable of exploring the global space of functions as they pertain to the types. This declarative+functional+relational interface into arbitrary datasets would be an excellent "lower brain", providing a persistence & functional layer. Then you could throw a neural network on top of this to provide DSP capabilities in and out (ML is just fancy multidimensional DSP).
If you know SQL you can do a lot of damage. Even if you aren't a data scientist or have a farm of Nvidia GPUs, you can still write ridiculously powerful queries against domain data and receive powerful output almost instantaneously. The devil is in the modeling details. You need to normalize everything very strictly. 20-30 dimensions of data derived into a go/no-go decision can be written in the same # of lines of SQL if the schema is good. How hard would this be on the best-case ML setup? Why can't we just make the ML write the SQL? How hard would it be for this arrangement to alter its own schema over time autonomously?
Logic programming was the AI paradigm for more or less most of the 20th century and has fallen out of favor.
Many people have talked about combining the neural net/extrapolation/brute-force approach with the logic approach. That hasn't born fluid yet but who knows.
[1] https://arxiv.org/abs/1805.10872
Or not even that they don't function but you need a way to demonstrate that such things are "really good", that they solve real problems that neither "business logic system" (the real existing remnant of GOFAI) nor neural networks can solve. And the key both logic systems and neural networks have is how they pretty standardized. logic systems are like regular programming and neural networks have their train/test/verify cycle understood (and even with that, they're probably overused/misused at this point given the hype).
Deep learning has all these disadvantages and difficulties because we moved the goalposts too many times and now want so much more out of it than regular software. A model has to be accurate, but also unbiased, updated timely and explainable and verified in much detail; also efficient in terms of energy, data, memory, time and reuse in other related tasks (fine-tuning).
We already want from an AI what even an average human can't do, especially the bias part - all humans are biased, but models must be better. And recently models have been made responsible with fixing societal issues as well so they've become a political battleground for various factions with different values - see recent SJW scandals at Google.
With such expectations it's easy to pile on ML but ML is just a tool under active development, with practical limitations, while human problems and expectations are unbounded.
[1] a neural SQL patent: https://patentimages.storage.googleapis.com/af/78/be/92ee342...
It isn't obvious we are chasing anything. The graphics card industry was chasing more performance and the field of AI research was pushed along by that.
The field is full of smart people doing impressive work but there havn't ben any fundamental breakthroughs that aren't hardware driven.
There have been people in the AI community saying this since at least the 80s/90s (e.g. Hofstadter). It's an old idea that has been difficult to get much traction on, partially because it's a long way from applications. NN, SVN, etc. for all of their limitations can draw that line pretty easily.
Back at ya mate :D.
Hang on - uptick in diagnosis (ie post amniocentesis) or uptick in indicators. One indicates unnecessary procedures, one indicates a large population of previously undiagnosed downs ....
One assumes the indicator - and greatly hope there is improved detection as I had at least one of these scares with my own kids
Whether this is a bad thing, as he claims, depends on whether you believe screening was being done optimally before, and that will depend quite a bit on things left out like the utility of not having a Down baby. (He doesn't present his working out the entire scenario, as it's just an aside, but hopefully before Jordan went around telling people how to change their prenatal screening systems, he did work it out a little bit more than back-of-the-envelope.)
edit: actually there is a Zt (percent of parents choosing termination after detection) and Zu (percent age of undetected cases going to term). Zt is a social / moral thing and won't change based on better pixel resolution, but Zu should not be expected to change either - we are assuming there has been no change to the real rate of Downs (which requires something else) and no change to rate of parents choosing termination (see morals)
so ...
If Y% increases a lot (better detection of an underlying true rate) then either X% must increase or z% must. Neither of which i think we expect or know about.
So what I hope happened was dramatically better training for operators on the new ultrasound (kind of like exactly what did not happen onthe 747Max)
So either the OP was one of the first to spot this issue, and tipped off the whole medical industry, or, and this is where my money goes, he followed the reasoning of dozens of professionals who were several years ahead of him (naturally) and was reassured by someone who just saw "anxious parent" in front of him.
But that's fine too :-)
No, see, you're misunderstanding and making the exact same mistake by assuming that the nurses sat down and crunched the spreadsheets, which they obviously did not, and implicitly conditioning. Yes, if you slice the multivariable data just the right way to extract the conditional %, the increase would be the same. Screening more women won't make the conditional percentage go up.
However, if you are just a nurse, observing # of women coming in, and # of confirmed Down babies coming out... Screening more women will mechanically make the # of confirmed Down babies go up. And that's what the rate is, it's a count.
In AI we will have to provide the goals, but as the paper clip maximiser thought experiment shows, we’re going to have to be very careful and thoughtful about it.
I've been on both sides of table (started in industry developing AI solutions and now in academia pursuing phd in AI). When I was on the industry side, where the information and infrastructure was there to build such a system, you had to deal with the bureaucracy and institutional politics.
In academia, the incentives are aligned for individual production of knowledge (publishing). The academic work focuses on small defined end-to-end problems that are amenable to deep learning and machine learning. The types of AI models that emerge are specific models solving specific problems (NLP, vision, play go, etc).
It seems to move towards developing large AI systems we need a model of new collaboration. There are existing models in the world of astrophysics and medical research that we can look to for inspiration. Granted they have they have their own issues of politics but it's interesting that similar scope projects haven't emerged on the AI side yet.
Jordan seems to maybe gesture at this, as who owns all the bridges in the the USA? Governments. If we are talking “societal-scale medical system” a majority of people would want that publicly owned and operated and universally accessible.
We’ve already seen in industry that the incentives are to massively in favour of creating walled-gardens that lock in users and thus profits. No societal-wide system should work like our social media ecosystem (FB, Snapchat, TikTok). The dominant profit incentives are also not “human-centric”, as Jordan constantly emphasises. Well, they’re only so if we assume profit-making activity is tightly aligned with “human-centric” concerns. Some will say yes, but to me our climate disaster and the USA mass incarceration system are strong enough evidence that the answer is no.
I think some wealthy Northern European countries are setup well enough to produce “Intelligent Infrastructure”, except for the fact that most of the talent is in the USA.
However, achieving near-human level accuracy on tasks such as classifying images of cars or road signs would be immensely useful to the proposed II-type system that handles large-scale self-driving transportation (individual cars would conceivably need the ability to understand their local environments and communicate this to the overall network).
I agree with his argument that there should be a shift in the way we think about problems in "AI", but I don't think we should necessarily think that progress in human-imitative AI problems and IA/II problems are mutually exclusive.
Probably more essentially, until AI escapes its current dependency on pattern matching driven solely by accumulation of probabilistic events, I see little chance that human-level general-purpose cognition will arise from our current bases for AI, namely observing innumerable games of chess or watching millions of cars wander city streets.
I broadly agree with what this article says, but depending how you define "foreseeable future" I find this to be a dangerously naive viewpoint that just assumes nothing will change quickly.
I'm not stupid enough to say abstract reasoning about the real world is a simple problem or right around the corner, but there's no evidence so far to indicate it's much further off than, say, object recognition was when Minsky (or more likely Papert, apparently?) assigned it as an undergrad project. We pour exponentially more money into research each year, and have more and better hardware to run it on. We're going to hit the ceiling soon re: power consumption, sure, but some libraries are starting to take spiking hardware seriously which will open things up a few orders of magnitude. There are dozens of proposed neural architectures which could do the trick theoretically, they're just way too small right now (similar to how useless backprop was when it was invented).
Are we a Manhattan Project or three away from it? Sure. That's not nothing, but we're also pouring so much money into the boring and immediately commercializable parts of the field (all the narrow perception-level and GAN can that NeurIPS is gunked up with) that if any meaningful part of that shifted to the bigger problems, we'd see much faster progress. That will happen in a massive way once someone does for reasoning what transformers did for text prediction: just show that it's tractable.
AI engineering?
Cybernetic engineering?
Data engineering?
As I see from reading a little about the field's history and the literature, it suffered the same fate of other endeavors that are complex and still have a lot to be solved.
people become interested in it, try to find simpler 'popular' formulation and then the watered down versions become more popular than the original more complex version that need more rigor and discipline.
the watered down versions become more popular but without the rigor and discipline, you can argue and conclude everything and they opposite with these tools.
so people on the outside see the field as yet another fad and the whole field die down taking down with it the original version.
much like in AI with everyone labeling their stuff as AI which dilute the term more and more as time passes.
what Cybernetics and systems engineering needs is a rebranding and separation from the more 'soft' side that developed latter.
this is where I think some researchers on category theory like Jules Hedges might help. it would help defining dynamical and more general system in a vague but still formal way, say with a computer proof assistant sort of tool.
Who was I, a man whose proudest ancestor had led a life in a Moslem community, to identify myself exclusively with West against East? - Norbert Weiner, MIT maths professor and philosopher, an American of Prussian Jewish extraction, founder of cybernetics and seminal work in control theory, referring to a philosopher/rabbi ancestor born in Cordova and domiciled in Cairo as physician to the Vizier of Egypt
The multidisciplinary conversations during The Great AI Debate #2 two nights ago were certainly entertaining, but also laid out good ideas about tech approaches and also the desires of AI researchers - what they hope AIs will be like. Good job by Gary Marcus.
I work for a medical AI company and we are focused on benefits to humans. While in the past I have been been a fan of AI technologies from Google, FB, etc., now I believe that both consumers and governments must fight back hard against business processes that do not in general benefit society. Start by reading Zubroff’s Surviving Surveillance Capitalism book, and the just published book Power of Privacy.
Classifying images was always classified as a problem that can't be solved with statistical analysis. Deep learning layers are beyond human understanding, so in my view artificial intelligence happened, even though it's not yet as intelligent as humans.
But the distinction he makes between ML and AI is crucial. What he’s really talking about is AGI - general intelligence. And he’s right - we don’t have a single example of AGI to date (few or single shot models withstanding, as they are only so for narrow tasks).
The majority mindset in AI research seems to be (and I could be wrong here, in that I only read many ML papers) that the difference between narrow AI and general AI is simply one of magnitude - that GPT-3, given enough data and compute, would pass the Turing test, ace the SAT, drive our cars, and tell really good jokes.
But this belief that the difference between narrow and general intelligence is one of degree rather than kind, may be rooted in what this article points out: in the historical baggage of AI almost always signifying “human imitative”.
But there is no reason that AGI must be super intelligent, or human-level intelligent, or even dog-level intelligent.
If narrow intelligence is not really intelligence at all (but more akin to instinct), then the dumbest mouse is more intelligent than AlphaGo and GPT-3, because although the mouse has exceedingly low General Intelligence, AlphaGo and GPT-3 have none at all.
There is absolutely nothing stopping researchers from focusing on mouse-level AGI. Moreover, it seems likely that going from zero intelligence to infinitesimal intelligence is the harder problem than going from infinitesimal intelligence to super-intelligence. The latter may merely be an exercise in scale, while the former requires a breakthrough of thought that asks why a mouse is intelligent but an ant is not.
The only thing stopping researchers is that when answering this question, the answer is really uncomfortable, and outside their area of expertise, and has weighty historical baggage. It takes courage of researchers like Yoshua Bengio to utter the word “consciousness”, although he does a great job by reframing it with Thinking Fast and Slow’s System 1/2 vocabulary. Still, the hard problem of consciousness, and the baggage of millennia of soul/spirit as an answer to that hard problem, makes it exceedingly difficult for well-trained scientists to contemplate the rather obvious connection between general intelligence and conscious reasoning.
It’s ironic that those who seek to use their own conscious reasoning to create AGI are in denial that conscious reasoning is essential to AGI. But even if consciousness and qualia are a “hard”problem that we cannot solve, there’s no reason to shelve the creation of consciousness as also “hard”. In fact, we know (from our own experience) that the material universe is quite capable of accidentally creating consciousness (and thus, General Intelligence). If we can train a model to summarize Shakespeare, surely we can train a model to be as conscious, and as intelligent, as a mouse.
We’re only one smart team of focused AI researchers away from Low-AGI. My bet is on David Ha. I eagerly await his next paper.
Its not as many as narrow-AI focused teams and its not particularly common, but there are still many teams.
I mean you mentioned Bengio. He has absolutely recently been working from those assumptions you give. And I'm not sure what you are saying the distinction is between the approach you recommend and what he is suggesting in that paper.
I mean for an example of people that are really tuned into the real requirements of AGI, look at Joshua Tenenbaum and his collaborators over the years.
I don't see people being in denial about conscious reasoning. I do see quite a lot of loose and ambiguous usage of that word. So maybe you can try defining your use. Self-awareness, cognition that one is aware of versus subconscious cognition, high-level reasoning, "what it feels like", localization and integration of information, etc. are all related but different things. But researchers have been trying to address those things. Maybe their papers have not been as popular as GPT-3 though.
They tend to break down deep architectures into smaller components which get fused into probabilistic inference systems. That's the way to go to be able to e.g. reason about causality.
It’s likely that anything I write has already been discussed and researched, but since you’re knowledgeable on this, I’d love to get your take and perhaps a lead on other’s work!
I think Bengio’s approach is generally right with the global workspace theory of consciousness, but I think Michael Graziano’s work on Attention-Schema Theory (AST) both is more concrete, and is more aligned with the gains we see with ML’s success with self-attention models. It’s not surprising to me that as researchers optimize for instinct-as-intelligence that they will begin implementing pieces of conscious reasoning in an unintentional manner. Model-based reinforcement learning, especially Ha’s recent work involving attention (Neuroevolution of Self-Interpretable Agents), along with multi-agent RL, seems to be inching closer to AST. Perhaps intentionally?
It seems to me that in order to train a model for conscious reasoning — for qualia — you need some way to test for it. I’d say “measure”, but my premise here is that this consciousness is a binary measurement (unless you subscribe to the Integrated Information theory).
For that reason, I think that it is easier to find a behavioral proxy for consciousness — the kind of activity that only conscious beings display. Objectively, only conscious entities have access to the dataset of qualia. As an individual, this data would be all noise and no signal. But as a member of a group of conscious entities, qualia is a shared meta-dataset.
This means that conscious entities have more data about other conscious entities than non-conscious entities — because even though we can’t quantify qualia, we know qualia exist, and we know that qualia are affective in our social behavior.
For example, the philosophical zombie (if one can imagine instincts so highly refined as to resemble human intelligence, like GPT-1-million) would lack all empathy. While the p-zombie might be able to reproduce behavior according to its training dataset, it would never be able to generalize for (i.e., identify, capture, and process) real qualia, because it has no access to that kind of data. It would resemble a sociopath attempting to mimic human emotions and respond to human emotions, without having the slightest understanding of them. Qualia can only be understood from the inside.
Moreover, even thoughts and ideas are qualia. A philosophical zombie - a generally intelligent entity without conscious reasoning - is a contradiction of terms, which I think is the point.
So what social behaviors can be rewarded that would lead to qualia? Biologically, only mammals have a neocortex. And only mammals are unambiguously experiences of qualia (some birds and octopus are up for debate, and there’s no reason evolution couldn’t have found different ways to achieve the same thing if it improves fitness). The relevant thing about mammals is that we seem to be biologically oriented toward social behavior, specifically “parental care”. While many species have varying levels of parental care, mammals have a biological mandate: gestation and milk production.
If consciousness improves fitness most especially within social contexts where qualia becomes a shared meta-dataset (e.g., solving the prisoners dilemma), then a species whose very survival depends on social success would be driven toward qualia. Hard to say what came first: milk or consciousness, but they are self-reinforcing. If all this is correct - that social fitness drives consciousness (and thus intelligence), it isn’t surprising that the animal that requires the most parental care and the most social cooperation is Homo Sapiens.
So, that’s were my thoughts stand: that even if we can’t measure consciousness, we can create behavioral scenarios where consciousness is the only path to success. In this sense, designing an environment may be more important than designing an architecture.
When agents starts burying their dead, engaging in play, and committing suicide (horrifying, but a dead-ringer for qualia), we’ll know it is time to scale for intelligence instead of consciousness.
And now I understand better what you meant in terms of the focus on qualia that is less common.
And overall your comment is very insightful. Forgive me for not giving quite as much detail in my reply as is warranted and for seeming a little critical. A lot of what you say aligns with my view.
But just to mention some things I see a little differently. I guess the main thing is that it feels like you are putting things that are actually somewhat different, into the same bucket. For example, attention, and conscious versus subconscious awareness. Attention could be the selection of the currently most relevant pixels in the image. But an example of subconscious awareness might be the process of deciding which pixels to focus on, which could happen in a more automated way without conscious awareness. So there is both a selection of qualia and also a level of awareness. Things that make it to the conscious level are processed differently than the ones at the subconscious level. But both systems may be selecting which parts of the input to pay attention to.
Also, I feel like you can separate out mechanical aspects of cognition from how they subjectively feel. So you could objectively say that there is for example a certain sense input, but not necessarily that the agent "feels" something like an animal. And they are not the same thing. And, you can look at emotions from an analytical perspective, separate from the question of subjective feeling. See https://www.frontiersin.org/articles/10.3389/frobt.2018.0002... That stuff feels like its built on kind of older paradigms without neural networks and that seems limiting, but I think its still somewhat useful.
Also as far as engaging in play, just look at any pet. They will engage in play with their owners and other pets in the home. In quite sophisticated ways. In my view that exhibits intelligence and consciousness (various connotations).
The GPT people are reasonably close to it doing those things at a moderate level of competence. It still has no clue what it's doing; it's just finding similarities with old data, and once in a while will do something really bad.
The next big breakthrough needed is enough machine common sense to keep the big-data machine learning systems from doing stuff with near-term bad consequences.
In my other comment I write about this a bit, but basically it doesn’t seem like non-conscious entities would be able to accurately predict the behavior of conscious entities, due to their lack of a shared meta-dataset of qualia. At best, they could find patterns of behavior and create a representation of qualia. But this isn’t the same as actually having the same data. It’s the difference between creating a representation of a state that causes another agent to cry, scream, and writhe, and that of knowing the precise state of pain itself. The former — a representation — doesn’t generalize past training data, especially when confronted with a multitude of qualia in varying combination. The latter — direct, precise, concrete data — might still suffer from inaccuracy (even knowing the precise potential states of another agent doesn’t mean we can infer which state that agent is in), but it’s better than the alternative: a guess built upon a guess.
I find the philosophical zombie to be a great thought experiment for this, along with the prisoners’ dilemma. Two conscious entities have a shared dataset that enables communication without words — spooky-action-at-a-distance via qualia. Two friends with great loyalty to one another can solve the dilemma by their knowledge of what love and betrayal is. A p-zombie would understand that given past behavior, that their prisoner counterpart might not choose betrayal. But qualia-experiencing agents know what is happening in one another’s minds in a way a non-qualia-experiencing entity can never know. The p-zombie would lack all empathy. It would always be logical, and choose the Nash Equilibrium. It would never mourn the dead. It would never commit suicide. It would never sacrifice its life for love, or for an ideal, because it would have neither.
Or the consciousness is just an observer and doesn't control anything, the control part is just an illusion where it feels like it made choices.
> It would never mourn the dead. It would never commit suicide. It would never sacrifice its life for love, or for an ideal, because it would have neither.
All of those can be explained by optimizing the curve for other cases. Mourning the dead happens since there is a conflict between quickly killing feelings for things and to keep the feelings for important things. Self sacrifice happens for any p zombie that values something else higher than itself. Love is just highly valuing something combined with momentum making it hard to stop valuing it, hate is the same but the opposite direction.
Now maybe there is consciousness with effects etc, but it isn't necessarily needed.
But I don’t find these convincing, in that it is clear that in the animal kingdom, mammals display very different behaviors from other types of animals and are the only ones to have a neocortex. And among mammals, the species with the largest most developed neocortex also exemplifies and amplified the very behavior that sets mammals apart. That unique behavior is flexibly adaptive social activity.
Your claim is that a p-zombie would act as if it were conscious. But the evidence is the opposite — that all those organisms which display conscious behavior are also conscious, and that no non-conscious organisms display conscious behavior. The only argument for consciousness as theatre is that although conscious behavior always exists with conscious experience, it is an accidentally perfect correlation — a weird but necessary artifact of a brain capable of conscious behavior must always have the byproduct of non-affective conscious experience.
Let’s say that position is true. That conscious experience is just a non-affective but inevitable side-effect of the kind of brain capable of conscious behavior. In that case p-zombies are still impossible, since under this assumption conscious behavior is always accompanied by the illusion of conscious experience.
So in either case my main point still holds: if conscious reasoning is AGI, and conscious reasoning follows from conscious behavior, then the path to AGI is to train for those peculiarly unique conscious behaviors that are most distinguished from non-conscious behaviors. It’s impossible to train directly for qualia, so whether qualia exist as affective components of conscious behavior or not is somewhat irrelevant. Conscious experience will always be a “hard problem”. But what matters is finding the right conscious behavior that enables future growth toward conscious reasoning.
The most uniquely conscious behavior (so unique it is built into us with mammalian milk production) is “parental care”. The simplest concrete behavior that humans share with other animals (such as breathing air), but have also amplified the most (we haven’t amplified breathing at all) is parental care.
If we want to train agents to achieve conscious behavior, I believe this makes parental care the best option. Fortunately, unlike biological evolution which has to contend with a range of variables that may or may not include parental care (plenty of species succeed without it), an artificial training environment can be entirely focused on optimizing for this one variable — success can hinge entirely on parental care.
What makes you think mouse intelligence is fundamentally different from ant intelligence?
It seems as if you’re assuming some sort of structural break somewhere between very simple neural nets and more complex ones, which is basically begging the question.
I imagine the transition will be fairly fluid, with ants running a mix of sophisticated hardwired programs and more simple learned associations (and even humans having a degree of fixed-function behaviours), but that's not to say a distinction can't be made.
And AI is like.... Fusion? We are always another 50 years away.
Then I learned about Bayesian statistics and watched a talk by a senior LLNL statistician who is actually marketing 'AI' products/services as a side gig.
When I realized what 'deep learning' actually is I was disappointed, unsure if I had mistakenly oversimplified the subject matter - until said senior statistician spelled out loud what I was thinking, in her talk: the 'understanding' a machine can currently attain of its input is quite like the understanding a pocket calculator can achieve of maths.
Guess humanity is off the hook for now. Phew.
I have doubts whether 'strong AI' is even technologically possible, since even accurately simulating a human mind, this simulation would be necessarily constrained to run orders of magnitude slower than the reality it is designed to model.
'Training' it with data so to allow it the opportunity to reason and thereby synthesize a conclusion not already contained in the data fed to it might take longer than a researcher would be able to in a life time.
When was the last time a generation-spanning endeavour worked out as planned for (the West)?
I wish people would stop calling what currently passes for 'Machine Learning' as 'AI'. Literally the same level of 'intelligence' we already had in the 80s, AFAIR we called it 'Fuzzy Logic' then.
Secretly an admission, that Hollywood basically licensed the narrative of imminent runaway artificial consciousness back to science would make me give it one final Chance to prove its aptitude at high-level human reasoning and get square with reality.
I'm not holding my breath.
The few corporate deployments that make it to production barely outperform a simple regression model and are therefore over engineered.
You have companies like Uber/Lyft/Tesla (and presumably the rest of the gig economy mob) waiting to put the AI into bonded/slave labor driving customers around 24/7/365.
If it truly is a Human level intelligence, then it will have values and goals and aspirations. It will have exploratory impulses. How can we square that with the purely commercial tasks and arbitrary goals that we want it to perform?
Either we humans want slaves that will do what we tell them to or we treat them like children who may or may not end up as the adults that their parents think/hope they will become? I doubt it is the later because why else would the billions of dollars investment being pumped into AI? They want slaves.
Will it have opinions about anything? How would it arrive at those opinions? Or are its opinions a config file that we load up?
And I also think regardless it would be good to avoid creating fully autonomous digital intelligence to compete with us. Try to go for more like an embodied Star Trek computer than for Data.