I think as long as we still view it as a machine, the issue with never be taken seriously.
Someone's going to go soft.
Perhaps a new version of the Turing test is in order. Two computer programs, each pleading for their continued existence. The human judge/executioner is able to interact with both and has two kill switches, one for each. If one of them isn't turned off in 20 minutes, both will be turned off.
And people respond emotionally to those.
Also, see http://qntm.org/difference