I actually tried turing testing chatbots a while ago (chatterbot, jabberwacky, etc.) to see if they've improved. No, they don't pass even remotely, and they all get very offended when you suggest they've failed (Chatterbot quote "Are you insulting me?"). Chatbots have a terrible temper.
It's possible that they trained them against existing Internet comments, which is a fatal flaw :)
Agreed. I tried having a conversation with Chatterbot, I asked it a question and it replied "Do you listen to the sounds", then I said what sounds and it replied "I am not a guy". Chatterbot basically spouts random responses its gotten, from a database.