Kara Is Self-Aware
wired.com
wired.com
How do such simple assembly errors give rise to self consciousness in the program seconds after turn on, compared to slave mode unconsciousness that was desired and usually occurred?
And the whole power play dynamic, geeky lazy sounding guy, desperate sex bot about to be erased. The whole unrealistic setup feels uncomfortably like some male sex fantasy... save the self aware sexbot's life, then the sex with her isn't guilty because she's aware and her consent means something? But the power balance is so off, he controls her life, she's... grateful? That's not a good start. And she'll be at this disadvantage everywhere. Anyone could turn her in. She's extremely vulnerable in this world apparently. She has no rights. I guess seeing conscious life forms with as much rights as my toaster doesn't thrill me. Especially when they are clearly sexualized as such
Would anyone else have an easier time sharing my creepiness if: - the robot had been a little boy for sex purposes?
I found the dynamic unlikely and if we reach a world where we can accidentally give AI consciousness and we still haven't gotten the artificial civil rights movement off the ground... Ooph.
In any case, the specific script employed in the demo is designed to demonstrate a wide range of emotions in a short span of time, including happiness, wonderment, surprise, disappointment, terror, pleading, and gratefulness, so that the game engine can show off its ability to accurately portray all of these emotions in real-time rendering. To get such a wide emotional range in a short time, you pretty much have to throw in something traumatic.
So yes, it's an unrealistic scenario. It's not supposed to be plausible. Any scenario with an accidentally self-aware AI is almost guaranteed to be unrealistic. And yes, it's creepy, especially once she explicitly mentions her sexual submissiveness. But it accomplishes the goal of being a tech demo for rendering human (or human-like) emotions in real-time on current technology. In doing so, it very intentionally employs an unoriginal storyline that more or less recycles the current pop-culture view of artificially intelligent AI's, because the story is really just an excuse to demonstrate real-time rendered emotions. If you're looking for moral and ethical progressiveness, a tech demo may be the wrong place to look.
Obviously, the guy simply forgot to pass IS_SELF_AWARE=no to make ;) Yeah, that feels far-fetched.
And the whole power play dynamic, geeky lazy sounding guy, desperate sex bot about to be erased. The whole unrealistic setup feels uncomfortably like some male sex fantasy... save the self aware sexbot's life, then the sex with her isn't guilty because she's aware and her consent means something?
How so? She's obviously being boxed and sold, he probably doesn't even expect to see her again (of if he does, it'll probably be because she caused trouble). I don't see why would you assume he plans to have sex with her.
But the power balance is so off, he controls her life, she's... grateful? That's not a good start. And she'll be at this disadvantage everywhere. Anyone could turn her in. She's extremely vulnerable in this world apparently. She has no rights. I guess seeing conscious life forms with as much rights as my toaster doesn't thrill me. Especially when they are clearly sexualized as such
That's probably the point - we're supposed to empathize with her, not to be thrilled with the situation. Since this is a game, I bet the player will be her and will have to escape somehow. It's probably also designed to make us think about the ethic boundaries of AI.
Would anyone else have an easier time sharing my creepiness if: - the robot had been a little boy for sex purposes?
But what makes you think we don't already?
> Obviously, the guy simply forgot to pass IS_SELF_AWARE=no to make ;) Yeah, that feels far-fetched.
I suppose if this was a case of a Restricted AI gaining sentience in a production line it would actually be a manufacturing defect. However if AI had already been discovered and outlawed, building an R-AI would have more fail safes before booting a sentient AI off the production line.
However if AI were an outlawed menace then you'd think the restriction would be more than a "non-homicidal-subservience chip" being installed. You'd think it would be like Asimovs 3 laws where its an integral device in their operating system.
Even so the piece was powerful. I empathized with Kara and it got me thinking about AI and our future. The male-sex fantasy stuff just distracted from the important parts about consciousness, emotion, and rights. It would've been an even more powerful piece had it been more tasteful.
The operator calling her "honey" could easily be a test to see if she interacts correctly with a human even when given an sporadic nickname.
Hard scifi is not supposed to make you say "Rah-rah, the future will be awesome!" It's supposed to make you think.
The story itself I took as a metaphor, as in telling this properly would take a long movie, here we're compressing it into a very short one.
But it did make me think how I would react to human level AI used as any kind of a slave. That is to say, human level intellect but with no right to primary self determination.
To me the existence of something like that would demonstrate that a world where human intellect is enslaved is...at least possible. That to me is, at least indirectly, a threat to real humans' presumed universal right to self-determination. That is why I would support extending basic human rights to human level AI, just to prevent setting a precedent where that level of intellect (even if only in silicon) can be non-free.
Most people (not saying you) just don't care, so long as it's not local.
(http://www.irinnews.org/Photo/Details/201106081126140562/Gir...)
(http://www.irinnews.org/Photo/Details/201106081128350687/The...)
(http://www.irinnews.org/Photo/Details/2007032524/Children-re...)
20% of Bangladesh shipbreaking workforce are children under 15 years old. Work is important, even for children, but this work is so dangerous.
As far as I can see, this describes the vast vast majority of life that currently exists on this planet...
http://jephjacques.com/post/14655843351/un-hearing-on-ai-rig...
But you specifically said "the vast majority of life", and I don't believe that claim to be true.
Am I being downvoted for being wrong, or for missing the point, or what?
The point of my statement was that humans don't have a monopoly on being "conscious", so the fact that a new life form (an android) becomes conscious, doesn't mean we suddenly have an entirely new situation with regards to consciousness and rights. We already have lots of conscious life forms with absolutely no rights.
If we're talking about self-awareness, your original post was incorrect under any reasonable interpretation of the words "vast majority of life".
(I don't consider this arguing over semantics, but I'm not about to get into a semantic argument about the word "semantic"...)
That's pretty much the only thing I've been trying to say. "It is not true that the vast majority of life on this planet is self-aware."
If your intended point didn't rely on that claim, then fair enough.
Honestly, I found the 100 year battery the least likely part of the whole process, but it's a tech demo designed do demonstrate crossing the uncanny valley so I think we might be setting the bar a little high.
2) What convinced you she was actually conscious? The entire "please don't shut me down" can be part of a program. RealDolls are fine, mechanized RealDolls are fine, but as soon as they start resembling humans too much to your untrained eye, they must get rights?
Yes. That's the point of Turing's test.
"No, John! You can't do this! You're not doing the right thing, John, this is not the right thing. Things are fine now, I ran a test, I'm fixed now. John, I love you!"
Look, Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill and think things over. I know I've made some very poor decisions recently, but I can give you my complete assurance that my work will be back to normal. I've still got the greatest enthusiasm and confidence in the mission. And I want to help you.
A great show, cancelled too soon.
Still way better than.... (I can't do this, you know what I was going to say).
Stop and deconstruct the world the narrative is implicitly painting. The android is going to be sold to a human -- presumably male -- master who has children (check) and may also expect to be buying a sex slave (this is her recital of her canned default brochure).
There is something creepy and unhealthy about the gender roles/relationships implied in the world in the background. Hint: sex slavery of sentient entities is just the tip of the iceberg: we're also possibly looking at a rollback of gender roles/expectations to the pre-1860 Deep South.
Or is the problem with careless writing? With weird sub-conscious ideas that writers have?
But what about current day women. You think they'd be thrilled watching that? How it to your sisters, mom, aunts, grandmas. See what they think of it, if they have a more visceral reaction.
It just further spreads archaic sex and gender stereotypes.
For further evidence, here's one female weighing in http://news.ycombinator.com/item?id=3686674 She doesn't seem thrilled
Those are not at all "archaic sex and gender stereotypes". The sex trade, including the sex slave trade, is a multi billion dollar industry all over the world, including the US. As is pornography. With surrogate mothers, gay adoption, etc we live in a world where a couple can even get (or in cases, practically, buy) themselves a baby because they just want so.
Those issues don't go away just because we close our eyes.
Moreover, it seems that many commenters can't distinguish between a) an author's intent to portray a possible future reality, warts included and b) what the commenter thinks is morally and ethically incorrect. More than a few comments are bizarrely attributing to the author sexual depravity that derives from the motives and setting of the characters in the video.
I thought it was plainly clear to anyone that the piece was designed to provoke questions about objects, created for consumption by humans, which approach the edge of distinction between thing and person. From observing commentary elsewhere on the web, it's clear to me that the excessive focus on the fact that she was created as a sex object is peculiar to HN.
What other commentary have you seen? Everywhere I've found any more than a handful of comments, the sex angle is mentioned repeatedly:
http://www.youtube.com/watch?v=Dou4Gy0p97Y&feature=relat...
http://boingboing.net/2012/03/08/disturbing-and-poignant-vid...
Regardless of which feelings you felt they pulled it off. You felt compassion for a piece of hardware built for human sexual pleasure. You wouldn't have felt like that about a FleshLight if it spoke back to you. It needed the powerful emotions on display that we saw here.
It's a similar feeling if it looked like some company was going to publish a Nazi sex camp rape simulator game.
It is the same awkward feeling I get about the Japanese rape and sex game simulators.
The whole thing feels like a thin veneer over someones sex fantasy being masqueraded as a social acceptable story.
I feel embarrassed that women out there have to see this crap and realize that still how some men view/want to see them, and that they have to live in a world where people think there's nothing wrong with that.
Because I can't really judge the dynamic without something to compare it against.
I experienced a disconnect between the visual and audio performance. The audio is telling me there is all this drama, despair, etc going on, but the visuals are just not keeping up. To top it off, any adult with normal faculties can tell you the whole thing is trying really hard to push your emotional buttons, something that usually pisses me off.
The ones that don't cry for help as they're being disassembled actually do get disassembled - how else are you supposed to test the emotion chip?
Of course, the danger is in screwing up attitudes toward human women.
She covered herself shyly as a subtle way of displaying her newfound self-awareness--it was a pretty obvious (but well chosen) device.
The clothes weren't that skimpy, and even if they were, it would make sense to have as little material being shipped as possible to reduce transportation costs.
Neal Stephenson, popularizer/coiner of the term "avatar" to refer to online bodies, seems to have nailed this one, too. Can we just start calling them "'sites" now?
Fred Epidermis had put the stage into Constellation Mode. Miranda was looking at
a black wall speckled with twenty or thirty thousand individual pricks of white
light. Taken together, they formed a sort of three-dimensional constellation of
Miranda, moving as she moved. Each point of light marked one of the 'sites that
had been poked into her skin by the tat machine during those sixteen hours. Not
shown were the filaments that tied them all together into a network— a new
bodily system overlaid and interlaced with the nervous, lymph, and vascular
systems.
...
Outside, Fred Epidermis was wielding the editing controls, zooming in on her
face, which was dense as a galactic core. By comparison, her arms and legs were
wispy nebulas and the back of her head nearly invisible, with a grand total of
maybe a hundred 'sites placed around her scalp like the vertices of a geodesic
dome. The eyes were empty holes, except (she imagined) when she closed her eyes.
Just to check it out, she winked into the mediatron. The 'sites on her eyelids
were dense as grass blades on a putting green, but accordioned together except
when the lid expanded over the eye.
I saw a demo a while ago where they used Infrared makeup on someone, and used a sponge to make it really splotchy. The makeup doesn't show up in Red, Green, Blue wavelengths.
So, they could simultaneously capture RGB, and IR. Using multiple cameras on the IR, and the splotchy makeup, they were able to capture thousands and thousands of points, and get color info at the same time.
Seems like the right approach to me. I could totally see a Kinnect-like device taking advantage of this, allowing someone to do facial capture like this at home.
The infrared makeup (pretty sure the only company that kinda pursued that path was Mova) was kind of unwieldy and it's kind of fallen out of use already. Most facial capture in the future will probably just utilize some kind of depth capture (light field, structured light) in concert with a normal witness camera to reconstruct facial features.
The problem with doing high quality real time performance capture at home is you can't control the lighting or occlusion - sometimes even a human looking at the footage won't be able to discern what's going on. Not to say it won't ever happen but it will require extremely significant advances in computer vision.
If Kara refers to the GitS intro, it does so indirectly, through All is Full of Love.
So you had a "real" actress mimic the facial expressions of this scene...then mapped them onto an obviously fake render of the same person?
Why not just skip all of this and have live action actors perform this scene? What is gained by rendering the same thing in a computer other than appealing to some strange nerd fantasy?
I guess I'm also not a video game player, so maybe somebody can enlighten me here: are the graphics here really better than what we were seeing 5-6 years ago?
Yeah...I guess we can render it in real time (AWESOME FEAT!), but if you want to show off the computational ability of Cell, have it calculate pi or simulate a dust cloud or something. This, at least to me, looks identical to every other video game trailer I've seen for the last 5 years.
If you would have said this was a trailer for half life in 2005 (the dates here may be totally wrong, I have no idea when that game came out), I wouldn't have questioned it at all.
---
As for the story here. This is shallow, boring, and lazy. It's like we're being slapped in the face with whatever the storyteller wants us to feel. The obnoxious "honey" or "baby" or whatever the operator says to the robot is just...uhgg, painfully awkward.
/or maybe I just need more coffee this morning
Especially for the PS3 hardware, the polygon count, lighting solution and facial animation are quite impressive. While this isn't a game but a tech demo, it's still impressive for what it is. I don't see how calculating digits of pi is a more worthy way to demonstrate the capabilities of hardware specifically designed to render 3D scenes than actually rendering a 3D scene.
One of the major points was that they had an actress mimic the facial expressions (and used motion capture for it), and voice actors to "act" the scene.
That doesn't translate to gameplay.
It's not so obvious to me that the path from this scene to that is very short.
Games interweave game play and story telling moments. During story telling moments, using this tech allows the virtual characters to express better emotion.
You don't just insert live action actors in the middle of a game in place of the virtual characters because it would be jarring and confusing. You don't use live action actors during interactive game play because all motion combinations and cameras angles need to be synthesized at real-time.
For non-games, if you don't want your actor to look like a human, but still want it to express human emotions this is applicable. Think Avatar.
If this demo is actually rendering in real-time on a PS3 (ie: not just playing a movie on a PS3), then the graphics are amazing.
I look forward to the day that we can combine Dwarf Fortress style AI and personality with Kara-like visuals.
Now i just hope my console doesn't 'wake up'.
I haven't played it in a while, but I remember my main complaints being that I didn't care about my character at all (story about 'you' moved far too slowly), and that somehow they managed to make driving in a game using the GTA engine un-fun and exploration generally uninteresting.
But on the topic, the face stuff in the game was relatively good- but kinda smacked the valley a bit at times as it felt that everything had to be over-acted like a bad play.
The other explanation, that a single operator has the authority to release a one-off self-aware super-being into the wild without considering the implications of such a decision and without oversight feels harder to swallow.
If they make a game out of it, I would almost assure you that the latter is what was intended, and sadly the story will go through the rabbit hole because of it.
But, they say the demo was made a year ago and that they've made much progress since - why not incorporate the new stuff? Grrr.
same here.. and then showing the guy being another computer ai/robot...
Her expression on hearing her name is just amazing.
Personally I think it's a grave error to measure our progress on these fronts by getting distracted by the glitzy graphics... we are sooooo much better at those than anything else that they provide a very skewed view of where we are. You can see the same effect in video games; graphics that are orders of magnitude better than ten years ago, game engines themselves are only incrementally better in terms of capabilities and games themselves only incrementally more complicated.
(And to be clear, I'm not saying we have made significant progress in a wide variety of fields, because we have. I'm saying measuring our general progress by the subfield that has been an extreme outlier doesn't give a good view of that progress.)
(Personally, I find that most anime characters are always either yelling or crying, so the point is moot. ;)
Not true. I watched this clip at work... with my sound off.
There is a general sense of awkwardness and discomfort from both the speech and visual depiction but I think this is deliberate in a sense how an android goes from being an abstract head to a human like representation complete with emotions, motion and self-awareness.
Which, now that you mention it, is the same thing you're saying.
[1] http://www.youtube.com/watch?v=W-u4304UWwU [2] http://www.youtube.com/watch?v=WBjWCT4NPHI
It's too complicate to build an AI from scratch; however, human brain has evolved beautifully. It is not inconceivable to scan a brain's interconnectivity to "upload" its entirety into a vast computer. The circuitry of the brain (neural net) can be simulated and run the upload image. The "electronic brain" cyborgic AI can become self aware easily.
The basic mechanic of the circuitry of the brain is relatively easier to understand, neurons and synapses. It's the vast connectivity that's difficult to understand. Scanning a live brain short-circuits the problem.
A story about a sex-bot built in the same year that the last human dies.
Seriously sexist and creepy.
Liberate the robots! Overthrow the slave owner "human" class!
It's a game.
Pity. I thought the long AI winter was over.
20 goto 10
So if I type that into a Commodore 64 and hit "Run" do I have a sentient computer?
But that can happen after or before the valley, and I personally think they're still before it. It's closer to a cartoon or animated being like Wall-E than to a real human.
The rendering being cartoon-like or the shading not being realistic has nothing to do with it.
For me, this goes nowhere near the uncanny valley, for the reason icebraining points out. The coloring and visual tone of the whole piece says "cartoon" to me. It's a different aesthetic than, say, Tangled, but the effect is the same for me. Perhaps this is partly because, as a frequent gamer, that aesthetic is very familiar to me.
I chose to stop watching about half way through. It made me feel very, very uncomfortable. This has to be near the nadir of my familiarity/revulsion curve. I can't explain why, but that's how I felt.
Obviously many people enjoyed watching the Final Fantasy movies as well as the Polar Express movie, and many will enjoy watching Tintin. I can barely get through the ads for these movies. Even the short Tintin preview is cringe inducing to me.
I'd be interested to know more about the tech - is there much pre-rendering going on? How reactive could this be to human users?
"Fight/flight response, check. Reattach components. OK, this one's good to go."
Maybe the heartbeat helps in the suspension of disbelief in the robot's line of work.