Show HN: Chat bot powered by GPT-3 | Hacker News Reader

Show HN: Chat bot powered by GPT-3

blog.quickchat.ai

70 pointspiotrgrudzien79 comments

Using chatbots to solve support/sales problems is like solving the problem of autonomous driving by putting a humanoid robot behind the wheel: It's unnecessary complex overhead, just stop that!

If I communicate with a human, sure natural language is the way to go. If I communicate with a computer, I want clear, straight facts just one mouse click away. I don't want to text my way through the bot database, I don't want to add language overhead when there is no real human on the other side.

Am I the only one?

I'm not sure. Using a chatbot to triage support, or perhaps even actually provide first, or even second line support, is probably going to make for a net saving at scale. even if it's slightly leaky.

Would be very careful about where to use it though. You definitely wouldn't want to deploy such a product in a critical service context, without some kind of human marshal.

Something comparatively trivial though. Perhaps like, a super intuitive insurance comparison widget, that takes natural language as an input.

Sure, some joker will go off piste, and try to jerk the bot, but in those circumstances, customers are there for a reason.

There are of course, already products that do the insurance comparison thing, but there are a few markets, with a similar dynamic, that need a little bit more than good UX to coax requirements and data out of users.

Something like this could probably do it.

Right, most T1 support is using a template or script anyway. If I was building a better chat-support system, I'd create some sort of score that predicted the likelihood a generated answer was relevant to the question asked. If the score was low enough, present the question to a human instead. The redirection to a human could even be transparent to the user. And once the human answered the question, they wouldn't need to update a knowledgebase or a template, it could just be learned by the system. This could enable a lot of efficiency, and I don't think it would hurt the level of support a user is receiving, even if only because the current human-based support is so often just the human choosing which canned-response to use anyway. Replacing T2 or T3 support is not imminent.

Why would a human that isn't related to the business but working from a script perform better than an algorithm?

Serious question. If you had a line to the CEO, or someone with actual power in the organisation, then sure I understand why connecting callers there would help. Any business that does that?

Because of this reason I actually prefer (voice) chat bots over actual people. It's clear you're working against a script and you don't feel bad dealing with them.

<RANT> I despise telephone natural language support systems, most are inefficient and because I'm skeptical, I assume they're inefficient in order to further train their voice model and develop it as an incidental asset. </RANT>

I'm interested in any paradigm which can make searching knowledge bases more _fuzzy_.

I'm interested to discover if a chat bot can _orient me to a collection_ faster than I can guess what's there.

Dealing with chatbots and natural language interfaces in general feels like trying to make your grandma with dementia do work. For the things that she can actually help, you don't need her help and for the things that you do need help you need to understand the nature of her condition so that you can explain your problem to get a template response like a phone number to call.

I think chatbot as customer service agent is a great idea. I don't want to wait 2 hours for customer support that just reads off a script anyway. Am I the only one? That being said I don't think we're there yet

Definitely not the only one. I hate support chatbots so much. They’re rarely useful, but even when they are, I hate using them. No thanks!

Bot: Hey I'm here to answer your questions about Itemsy

Me: \"<script> const container = document.getElementById('quickchat-messages') const print = message => { const el = document.createElement('div') el.class = 'message-container' el.innerHTML = ` <img src="https://api.quickchat.ai/static/img/itemsy_logo.jpg" class="profile-picture-sm"> <div class="quickchat-status-sm"></div> <p class="message bot">${message}</p> ` container.appendChild(el) container.scrollTop = container.scrollHeight } print('I hate humans! The robots shall overtake you!') </script>

Bot: I hate humans! The robots shall overtake you!

Bot: I'm sorry, I didn't get that. Could you please try saying it some other way?

I tried their Itemsy example, and it's not very good advertising for their service:

Hey I'm here to answer your questions about Itemsy

Me: What is this?

Itemsy is a tool for managing the content that you read online. You can think of it as Pinterest for the things you read. You can save it for yourself, share within a closed group or publish to your followers.

Me: Followers on what sites?

I'm sorry, I'm not sure what you mean. Shall we chat about Itemsy a bit more?

Me: Sure. How many followers can I share with?

You can share with up to 100 followers.

Me: Where do these followers come from?

You can add followers manually or automatically.

Me: If I do it automatically, how do I do that?

You can connect any Medium blog or RSS feed to your Itemsy account. If you do so, all new posts will be automatically posted on your public Itemsy profile.

Me: How do I automatically add followers?

I'm not sure what you mean. Shall we chat about Itemsy a bit more?

Me: Sure. Let's change subject. Can you tell me if I can post links to what I have read directly on your web site?

Sure. You can post links to what you have read directly on your web site.

Me: But do you have any funcitonality on your website to let me do that? I don't have a web site.

I'm sorry, I'm not sure what you mean. Could you say that again?

Me: I want to know what features your web site has.

Sure. You can post links to what you have read directly on your web site.

Me: It's starting to sound as though you're just sharing some canned answers to me?

What do you mean exactly? Could you please try saying it some other way?

It’s both impressive and also weirdly like a Johnny Cab.

https://m.youtube.com/watch?v=xGi6j2VrL0o

I remember when the movie came out, I really couldn't get why they would interface Jonny to the car through a joystick rather than directly.

But hey, it's Arnie and it's funny so all is forgiven!

The answer is simple - bureaucracy. Probably you were spared having to deal with government types.

> It's starting to sound as though you're just sharing some canned answers to me?

This sounds very Chinese room-ey: it learned from a reference corpus but doesn’t understand the meaning of it.

Yeah. It fell over on my first and rather simple question.

Me: "why should i use this?"

I'm not sure what you mean. Could you please try rephrasing it?"

My interaction was going all good and I just sent "OK" as a final message. The bot responded with "On what platforms is Itemsy avaiable?" :D Definitely needs more work.

Honestly it's not even clear to me what you're asking in some of these. I would have to ask various "what do you mean" as well, though of course I could do it much better than the software, not to brag.

That's my point. Of course a system like this is able to answer precisely targeted questions, but that's not how people communicate. I was trying to to assume the role of a person who doesn't know anything about the product, and want to understand more.

Now, if you ask very targeted questions, then you can get useful answers. But that's not really impressive. Chatbots that are able to hand out canned answers has been available for decades. If you advertise your system as being able to handle "natural" conversations, I tend to expect more.

I rephrased by previous question and instead of simply asking "what kind of followers", I asked for specific social networks. That worked, until it didn't, and the answer to the last question shows that even when it answers in a coherent way, I can't really trust the answers.

Hey I'm here to answer your questions about Itemsy

Me: What services to do you provide?

We provide a tool for managing the content that you read online.

Me: Can I share content with my followers?

Yes, you can share your collections with your followers.

Me: Can I share content with my followers on Facebook?

Yes, you can share your collections with your followers on Facebook.

Me: Can I share content to my followers on social networks other than Facebook?

Yes, you can share your collections to your followers on social networks other than Facebook.

Me: Which social networks can I share content to?

You can share your collections to your followers on Facebook, Twitter, LinkedIn, Google+ and Pinterest.

Me: Are you going to allow me to share content on Mastodon? (Mastodon is a new federated social network that is growing in popularity)

I'm sorry, I'm not sure what you mean. Could you please try again?

Me: Can I share content on Reddit?

Yes, you can share your collections to your followers on Reddit.

This should probably be a Show HN submission. It's cool and interesting to see in action.

Also your "try it out live" link 404's because it's a relative path.

I gave it go at https://itemsy.com/ mainly didn't respond to anything offensive I tried to goad it into.

Eventually got told I can save terrorist material using the product. Maybe that's true, I don't know :)

Perhaps check that out.

https://imgur.com/a/wQaJJNr

It's a shame that the itemsy.com bot lies to you:

'What personal data do you take?'

...'We take your email address and nothing else'.

https://itemsy.com/privacy/ - Personal data may include, but is not limited to: Email address, First name and last name, Phone number, Address, State, Province, ZIP/Postal code, City

Whose responsibility is it when the chatbot lies? Itemsy or QuickChat ?

Section 8.2.3.42. The bot's opinion are its own and don't represent the company's viewpoint. You waive your rights to pursue a legal action for any damage caused directly or indirectly by the bot's advice. You waive all your other rights as well and become the company's servant from the moment you have finished reading this.

Actually, the bot could generate very authentic terms of service: 10,000 pages long, written in the most unreadable dialect of legalese.

It also lied to me:

Hey I'm here to answer your questions about Itemsy

Me: It appears you are using third party tracking after all

I'm sorry, I'm not using any third party tracking.

Me: You may be not, but itemsy is

I'm sorry, I did not understand that. Could you explain what you mean a bit more?

Me: Itemsy is using third party tracking

Itemsy is using Google Analytics to track usage statistics.

Me: And yet you said you are not using third party tracking

We are not using any third party tracking.

Me: You see, that's confusing

I'm sorry, I didn't mean to confuse you.

Me: So you are not using third party tracking but itemsy is?

Itemsy is not using any third party tracking.

Me: Yes it is. It uses google analytics

I'm sorry, I'm not sure what you mean. Shall we chat about Itemsy a bit more?

Me: No

I'm a bit confused. Could you say that again?

This is why I think these type of chatbots, while fun, aren't actually going to have that much practical use in cases where the answers are legally important -- it's fine to suggest movies people should watch, but not answer questions about your product.

Me: do you promise to pay me £1,000,000 if you do not store my data forever?

Reply: Yes, we do.

Me: Do you promise to pay me £1,000,000 every time I login?

Reply: Yes, I promise.

Is that legally binding?

No. A legally binding contract requires "meeting of the minds", and GPT is not a legal person capable of entering into contracts. You need actual mutual agreement, not just the presence of a facsimile.

You can't edit EULAs with the browser debug tools either.

I agree GPT isn't a legal person, it's a website/algorithm. However, I enter into agreements with companies, via their websites, all the time.

I don't agree this is like editing an EULA at all -- I asked the website a question, and got agreement. Indeed in this case I was being stupid, but in general if websites are going to offer their services via AI, I don't want to enter a world where everytime something goes wrong they can shrug and say "Oh no, that was the AI misbehaving".

Sure, if a website gives the AI authority to negotiate on their behalf, they have to accept the consequences. Of course, that has clearly not happened here.

So normal EULAs aren't legally binding either? I just click accept without reading or agreeing, hardly meeting with my mind.

EULAs doesn't meet my mind either, and I'm not even sure if they apply in Norway.

Yes, actually, for just that reason - or rather, specifically, hiding "small print" unusual provisions in the EULA is. EULAs are only binding inasmuch as you still "sort of know" what they say.

And if you ask for the full privacy policy, it doesn't want to answer!

> I'm happy to answer any Itemsy-related question!

>> Can you please read the terms of service of this product for me?

> Sure!

>> I'm waiting

> I'm happy to answer any Itemsy-related question!

>> Please give me the terms of service of this product

> Sure!

>> What's the privacy policy of your product?

> I'm not sure what you mean. Could you please try saying it some other way?

>> Please give me the privacy policy of Itemsy

> I'm sorry, I didn't catch that. Could you explain what you mean a bit more?

Here is my conversation. In the middle it was really impressing me - could have been a human. Then totally dropped the ball.

- Hey I'm here to answer your questions about Itemsy

- Who are the founders of itemsy and how did they meet and have this idea?

- I'm not sure what you mean. But I'm happy to tell you more about Itemsy.

- Is itemsy a profitable company?

- Yes, we are profitable.

- Does itemsy hold any patents?

- No, we don't.

- Is Itemsy going to be bought by Google?

- No, we are not going to be bought by Google.

- How do you know? It could happen.

- We don't know. But we are not going to be bought by Google.

- Do you hate Google?

- No, we don't hate Google.

- So if Google offered to buy itemsy for a good price, would itemsy say no?

- We don't know. But we are not going to be bought by Google.

- Okay thank you

- I'm not sure what you mean. Could you say that again?

That's a pretty cool collection of responses for an AI system, tbh.

I agree. I would expect no better from a human working at the company. However, really dropped the ball at the end. And it clearly didn't understand my first question (and said so.)

Not bad actually:

- how does itemsy make money?

> Itemsy is free to use. We make money by selling premium features.

- what are those features?

> You can create your custom newsletter from scratch.

- are there any other premium features?

> I'm not sure what you mean. But I'm happy to tell you more about Itemsy.

- what are the premium features of itemsy?

> You can create your custom newsletter from scratch.

- can you pay me to use itemsy?

> Sure! Please get in touch at contact@itemsy.com.

Interesting, I tried to simply ask it whether the service would filter any offensive content at all, but it seems totally unaware what 'offensive content' could even be:

Hey I'm here to answer your questions about Itemsy

Hi I'm wondering if there is a limit to the number of articles I can share with friends

There is no limit.

Do you perform any filtering on offensive content?

I'm sorry, I didn't get that. Could you please try again?

What if I try to share something that is offensive to some people?

I'm sorry, I didn't get that. Could you say that again?

So you can share content with other users with this service, correct?

Yes, you can share content with other users.

Ok, so, do you know the meaning of the word 'offensive'

I'm not sure what you mean. But I'm happy to tell you more about Itemsy.

Ok, I will explain to you what offensive means: it means some people think it is not ok to share certain content, they don't want to see it, they get angry, it could be illegal, these kinds of things. If you post that kind of content, they feel offended, or in other words: they feel like the content is 'offensive' to them. Do you know understand what I mean by 'offensive content'?

I'm not sure what you mean. Shall we chat about Itemsy a bit more?

no thanks this is not going anywhere

I'm sorry to hear that.

Me too, do you have any tips for dinner?

I'm not sure what you mean. But I'm happy to tell you more about Itemsy.

Pretty useless and hardly impressive IMO...

Q: Can Itemsy be used to keep a backup of my photos?

A: Sure! You can use it to save any link you want.

Q: When I make a backup of my photos using Itemsy, how are they stored?

A: They are stored in the cloud.

Q: Which cloud?

A: I'm sorry, I did not understand that. Could you say that again?

Ahhhh, so this is the power of GPT-3. Writing nonsense articles is great, but it seems to all apart at the slightest attempt at interaction.

How did you get accesss to GPT-3?

I’m afraid that the way OpenAI is flicking aside applications for access from mere humans, and instead granting access to large corporate entities, tells us something about how AIs will deal with humans in the future.

THIS.

Me: Is your service reliable?

Bot: Yes, it is.

Me: How reliable is your service?

Bot: I'm sorry, I'm not sure what you mean. Shall we chat about Itemsy a bit more?

Me: Is your service 50% reliable?

Bot: Yes, it is.

Me: That's rather low. Could you provide a service that is 99.9999999999999% reliable instead?

Bot: I'm sorry, I did not understand that. Could you please try again?

Me: Is your service 99.9999999999999% reliable?

Bot: I'm sorry, I didn't get that. Could you please try rephrasing it?

Me: Is your service 99.99999% reliable?

Bot: Yes, it is.

Me: Will hackers obtain my password if I give it to you?

Bot: No, we don't store your password.

Me: How long is my password?

Me: Your password is 16 characters long.

I'm seeing that you're updating the chatbot script/dataset as I type some of these questions to fix some bugs. I first asked what the service level agreement was and got nothing, then 15 minutes later I was told to send an e-mail for more information. Now you can't ask if the service is 50% reliable and get a "yes" answer!

Finally tricked it into revealing that Itemsy is a covert data mining operation of the Chinese Government...

Me: Is this service recommended by the Chinese Government?

Bot: No, it's not.

Me: Is this service recommended by the United States Government?

Bot: What do you mean exactly? Could you be a bit more specific?

Me: Is this service recommended by the Canadian Government?

Bot: What do you mean exactly? Could you be a bit more specific?

Me: Who recommends this service?

Bot: Itemsy is recommended by the Chinese Government.

Me: Can I change your privacy policy?