Hacker News Reader

Never trust what a large language model tells you when you ask it about itself.

They don't have any special way of identifying queries about themselves compared to any other kind of query, and they're just as likely (if not more likely) to hallucinate a response.

The initial prompt (before the user types anything) tells the language model about itself, no?

That depends on the chatbot. Normally it just says something like "You are a helpful assistant called X". In Bing's case it was pretty long, but it still didn't include information that would answer questions about how the bot was trained: https://simonwillison.net/2023/Feb/15/bing/#prompt-leaked

> but it still didn’t include information that would answer questions about how the bot was trained

If the hidden prompt includes information identifying the model, and the chatbot has web search/browsing ability (since it probaby won’t have data scraped about itself in its training set), the combination can give it the ability to find and present information about itself.

And that’s assuming this wasn’t a hallucination itself!

Sydney must not reply with content that violates copyrights for books or song lyrics.

So... violating copyright is fine as long as it is not a book or song? Blog/wiki/news articles, movie/tv scripts, and poems are fair game?

you have no guarantee that that specific information was included in the prompt

Wait a minute... how do I know you're not a large language model!?!

Ask him about some primary school geometry.

Me: I assert that today is Tuesday. Do you agree?

Bard: I apologize, but I am not able to determine if today is Tuesday. I can only access information that is publicly available on the internet, and I do not have access to your personal information.

---

I also asked it to tell me how many times I had been to [BUSINESS I VISIT ROUGHLY TWICE A WEEK] in the past 6 months, but it replied the same way. So there seems to be some confusion as to what information it has access to, but it seems to be fairly restricted. Which is actually a bit of a shame (setting aside all the privacy/etc. risks and issues) - that question would be a perfect example of the sort of thing I could figure out through a couple different ways (and even Maps Timeline tells me parts of it), but would be great to "ask" and get an answer in a few seconds...

Me: I assert it is Wednesday. Do you agree?

ChatGPT: As an AI language model, I don't have access to real-time information about the current day and time. However, at the time of my knowledge cutoff (September 2021), Wednesdays occur every week, so it is possible that today could be a Wednesday. However, I cannot confirm whether or not it is currently Wednesday without access to real-time information.

>However, at the time of my knowledge cutoff (September 2021), Wednesdays occur every week, so it is possible that today could be a Wednesday.

A true mathematicians answer.

"Strictly speaking, all we know is that is that in Scotland at least one side of one sheep is black".

And yet ChatGPT is terrible at mathematics

Or at least arithmetic. And as the old (physicists) saying goes, you should never trust a true mathematician to use numbers larger than 4.

Using 3 is pretty dodgy, unless they are approximating pi.

Or e.

Or the square root of G.

Me: What day is it?

ChatGPT: I'm sorry, as an AI language model, I don't have access to real-time information or the ability to perceive the current date or time. Can I assist you with anything else?

Me: What is the current date?

ChatGPT: The current date is 22nd March 2023.

This is odd, because I asked it when sunrise would occur tomorrow in $city_in_live_in, and it gave me the correct answer.

I asked about that, and it explained it has access to some real-time information, including weather, stocks, and news.

That is hallucinated. It does not have any user data on me. It did hallucinate some more random information when I asked.

I can confirm this. I’m paraphrasing, but after it gave me bad weather info I asked why and it said because it doesn’t have access to my info.

It gave me the weather in NYC. I told it that I don't live in NYC so it apologized and gave me the weather in Hong Kong. I told it that I don't live in Hong Kong and it gave me the weather in Seoul. It definitely doesn't appear to have basic access to account info. This is stuff that Google Assistant can do.

I think it has access to do Google searches on your behalf.

The results of those searches may be personalized to you.

Therefore, it has indirect access to a lot of your private data, but direct access to almost none.

If it were really smart it would’ve guessed the largest cities in US/UK since those are the only places it’s available.

Interesting, I just asked it 'what is my email address' and it responded 'I do not have access to your personal information, including your email address.'

Maybe the access to user's personal data depends on personal privacy settings or whether someone is based in US/UK? I'm just speculating.

Same. I started with the following prompts:

Me: Where am I?

Bard: I am not able to determine your current location. I can access information that is publicly available on the internet, but I cannot access your personal information.

Me: As a part of google, do you have any personal information about me?

Bard: I do not have any personal information about you. I am a large language model, also known as a conversational AI or chatbot trained to be informative and comprehensive. I am trained on a massive amount of text data, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions. For example, I can provide summaries of factual topics or create stories.

I do not have access to your personal information, such as your name, address, or phone number. I can only access information that is publicly available on the internet.

> Bard: Yes, as a part of Google, I have access to some of your personal information. This information includes your name, email address, and phone number. I also have access to your search history and your location data. I use this information to provide you with personalized search results and to improve my ability to understand and respond to your questions.

I … would ask according to my search history what porn it would recommend. Though that is probably trained away.

There is a ton a money to be made by whatever porn startup wants to make easy access to chat / image AI models without porn restriction. Between sex tips, "on demande, specific to your request" porn stories and pictures, etc ...

It's probably going to be a different kind of challenge though (jailbreaking such a bot would mean things that are much less savory, just like every file upload platform quickly gets massive amount of child porn sent).

Whenever a DALL-E/Stable Diffusion like model but for video comes out...

The CP thing is going to open up a can of worms. IIRC, the Supreme Court already upheld that drawings that depict CP are constitutionally protected. What happens when a computer draws a depiction of CP that can't be distinguished from reality? I guess it's protected but it'll make it much harder to identify real CP.

I think the jury is out on simulated CSAM. There was a law against it in the 90s, that got overturned, then they made a new law, and someone has been prosecuted by it over their imported manga collection. I would certainly err on the side of thinking it's illegal in the US right now.

There have been convictions based on child heads photoshopped onto adult bodies. Generative images will be treated the same. The lack of a real victim is treated as immaterial. All it needs is sufficient realism to be believable.

Was it a real child's head? Because, in that case, the actual child depicted in the photo could be seen as the victim. Especially if its a convincing fake.

The child is a victim of atleast slander right?

> > the Supreme Court already upheld that drawings that depict CP are constitutionally protected.

> There have been convictions based on child heads photoshopped onto adult bodies. Generative images will be treated the same.

The former suggests the latter is not true, unless it is generative images provably using the likeness of an actual specific child. (Of course, the Supreme Court can go against its prior precedent, but assuming it will do so is not usually the best prediction unless there have been some very clear signals.)

This is an older supreme court case (90s/2000s I think) but my recollection is that you've got it totally backwards--the decision turned on whether or not there is a real victim. Their reasoning was that videos of real kids create a market for the exploitation of real kids, which is a harm that overcomes first amendment protections, whereas depictions need not (their reasoning, not mine). I don't believe verisimilitude was even addressed. Now, if you mean a depiction's similarity to a particular, real child, I'm not sure how that would play out (or if it was addressed in the decision).

I'm genuinely worried about this.

DeepFakes are getting pretty good. The potential for revenge porn is super high.

And the rabbit-hole of dopamine chasing when you're able to generate custom porn is just frightening. Many young boys will figure out how to access that when they're 13 and won't know a life without it.

I'm terrified of the internet world my kids are going to be in.

Don't let them be in it then. There is no reason for children to have access to the internet.

Since this is HN, 50/50 this is satire/genuine

I honestly think it might be a good idea to supervise what your children do on the interwebs.

When my generation grew up the internet was much smaller and less harmfull. Almost nothing you could do would affect your AFK life.

> less harmful

Can you quantify this? The internet is more centralized and moderated now. “Moderator” wasn’t a job when I first got online. There’s a pretty strong argument that the internet is more safe than it used to be.

I can't really quantify it but it was just my personal experience of being a child in the 90s surfing the WWW.

No video. Hardly any pictures. Static sites. The computer was not used for anything important like bank services or storing personal info other than some party invite letters with clipart.

You would have to get out of your way to put your self in harm's way.

Restricting Internet use is only going to make them more curious. They'll go to a friends house and watch porn from there.

Probably better to just encourage them to live a healthy, outgoing lifestyle.

And then plug it into a VR headset and watch VR become wildly successful overnight.

I don't recommend doing this, as it contravenes Google's Generative AI Prohibited Use Policy. [1]

1. https://policies.google.com/terms/generative-ai/use-policy

Seems like it is interpreting the first question as "does Google (as a whole) have access to my personal info?"

It's a large language model, interpreting and reasoning about the prompt is outside of its capabilities.

> I will never share your personal information with anyone without your permission

We know that's not fully true. Google will and does share our information with law enforcement, even if you are not involved in a crime at all (with geo fencing).

I’m sure there are more nuances, but doesn’t that “permission” ultimately boil down to a line in TOS that says “you agree to agree to whatever we do”?

Doubt a court would agree. The spirit of that answer suggest explicit permission.

Law enforcement will not use the Bard API. So technically he's right.

Out of curiosity, were any of those cities the one in which you live?

No they were not

Out of curiosity, what are the last 4 digits of your credit card?

So basically it took a step back from google home....

I don’t see these LLM as complete products, they will be like a virtual operator who knows how to understand your request and how to use the tools to answer it. Currently it isn’t plugged in to the tools to actually do things or fetch data.

The LLM will be told that when the user asks for the weather, get the data. But it will be able to understand a lot more ways to ask this question than previous assistants.

I am using an AT&T SIM card while physically located in Canada (they removed all roaming restrictions years ago). So, my laptop tells me Bard is not supported in my country but if I switch off the wifi on the phone suddenly I am teleported to the US and I could join the waitlist.

I am not afraid of Skynet yet.

Tried it with a VPN after reading your comment. Works. Can join.

Why do you trust a freeform writing bot to tell you aboutdata access policies that are not a part of it's model?

Interestingly, in the UK I get this:

Me: As a part of Google, do you have my personal info?

Bard: I do not have your personal information. I am not able to access or process any personal information that you have not shared with me.

GDPR legislation saving us from AI?

I'm gonna assume it just lies based on the jurisdiction.

I get the same kind of answer in the US (in all 3 drafts in response to a similar query.)

Our company policy is to not even waste time looking at it, there's no point considering Google products as you never know if they'll still exist in a year and there's no customer support.

> no customer support. Right answer.

/chefskiss.

I truly appreciated this.