The threat isn't that robots will kill you, but that they will take your livilihood. If you are a worker in a capitalist society, you should be worried. The analogy is the internal combustion engine vs the horse. What happens when AI advances to the point of matching your general intelligence and learning capacity? If you work in IT, poof, firm management outsources your job to a computer. If the AI is attached to a robot that further matches your dexterity, poof there go most other jobs as well.
Jobs going 'poof' thru automation used to be a GOAL of modernization. Is the race going to work hard forever? Will we ever be free from the need to provide food and shelter? Wouldn't that be a good thing?
The real issue is, WHEN we no longer need to work to all have a great standard of living, HOW will we work it out.
If you're worried that a sentient robot is going to kill you, why are you not first worried about a non-sentient robot being able to kill you? If robots are in a position where they can kill, sentient or not, humans will exploit them to do this far before an AI is conscious enough to do so. There is plenty of volition for this from humans, e.g. terrorism.
I think the "high profile people" are not afraid of the robotic entity being nativity good or evil rather the affect the AI device has on humanity being detrimental to overall human advancement. For example, look at how smart phones have caused people to only be able to recall a handful of phone numbers.
Which has left us more capable of remembering other things, right?