Hacker News Reader

Hacker News

Comment by ninetyninenine | Hacker News Reader

Full thread

ninetyninenine

The biggest risk of ai can be simply characterized by one concept:

This very report could have been generated by AI. We can’t fully tell.

View on HN

Sure you can. You simply email one of the listed authors and ask them if the document is legit.

And if they say "yes, it is legit", what does that tell you?

That the author has risked their reputation on the claim. If you're doubting the author is legit, interrogate their professional associations with an internet search, relying on the domain name system.

Nothing about any of this is new or profound. Counterfeit documents have been around for hundreds of years.

The question you replied to wasn't "why should you believe someone who says they are behind a piece of research", it was about the usefulness of receiving an email saying it.

Their point (I assume) was that it would be illogical to worry that the report might be written and released by AI yet consider an email response as evidence against it.

If AI can create and release this report it can also hijack a real person's email or create a fake persona that pretends to be a real person.

Three people make a tiger[0], and even current LLMs are good at pretending to be a crowd.

[0] https://en.wikipedia.org/wiki/Three_men_make_a_tiger

There's a certain point where this line of thought just becomes an AI-themed rehash of "Trusting Trust" by Thompson

So what if it was generated by AI? Does that invalidate its contents in any way?

Interesting point. Are we already at a state where an A.I. could respond to email, phone calls, and even video calls in a convincing way ?

Email definitely, just have to remember to fine tune so it says "sure, I'll get on that after lunch" rather than "as a language model…".

Voice calls, yes: I attended a talk last summer where someone did that with an AI trained on their own voice so they didn't have to waste time on whatsapp voice messages. The interlocutors not only didn't notice, they actively didn't believe it was AI when he told them (until he showed them the details).

Video… I don't think so? But that's due to latency and speed, and I'm basing my doubt on diffusion models which may be the wrong tool for the job.

No it does not.

This is the scary part.

Do you mean scary as in "it is scary that apparently intelligent and sane people are wasting time and money on producing documents that consist purely of meaningless fluff"? If so, I agree.

Yes, but also scary that it can produce meaningful fluff that will move you and you can't tell the difference.

it could also be written by a communist/terrorist/nazi/russian in Notepad on Windows XP. you can't fully tell.

We can’t. The point is 2 years ago between ai and a human, we 100 percent could tell.

Now we can’t. In the future it will be even harder.

Reply on news.ycombinator.com