Hacker News Reader

Comment by thr0wawayf00 | Hacker News Reader

This is what concerns me. Many people worry about being replaced by AI, but I'm far more worried about AI completely polluting digital communication. I think stuff like this is going to make it harder for companies to assess candidates, fight fraud and generally operate. I hope I'm wrong, but thinking about AI entering the digital communications arms race is only going to make things more difficult, i.e. more personalized spam call scripts, more intelligently targeted ads/spam/scams.

I think AI is going to force us on some level to reclaim some of our communication to the physical world, if for no other reason than it's going to be one of the only ways we can truly verify that we're talking to another person.

View on HN

> I think stuff like this is going to make it harder for companies to assess candidates, fight fraud and generally operate.

Yikes. Now I imagine a totally unqualified candidate for literally anything feeding interview questions to the chat bot and I don't like it at all.

Even worse: now there is a justification for forcing candidates to solve coding problems on whiteboards, as interviews and coding homework problems will be considered intently suspect.

My single worst interview experience was an on-site five hour marathon of whiteboard coding, with a grumpy senior insisting that the code on the whiteboard be syntactically correct. Nothing screams "we want unthinking ticket crunching machines" like optimizing for candidates willing to focus on writing code by hand for hours on end.

Naturally, I rejected the follow-up interview, but I fear that more companies now are going to demand this nonsense.

Side note: in my personal example, the whiteboard session wasn't the reason I turned them down; I asked every person on the team roughly how many hours a week they worked and not one of them answered the question (instead redirected the conversation towards "you can work from home sometimes!" type answers).

Since then, however, I have rejected other companies trying to ape this style flat out. A short half hour, fine. Five hours? Pound sand, I say.

You know the real issue there? In 5 years that kind of company will be using only CodeGPT instead of hiring humans.

I think any company relying solely on AI to build a tech business in the future is itself at risk. Where's your moat if your business is built entirely on AI licensed from someone else?

In an era where that's possible, the expectation will be for humans to be working hand-in-hand with computers, whether to make superior code or answer better interview questions. The bar will simply be elevated, and you will have to judge candidates on their computer/human synergy. The only time that what you say could be a problem is if the technology to answer interview questions is far superior than that of doing actual work. But then there is the next round of interviews (in-person, etc.). This also kind'of exists today in the form of googling answers while on interview, which for some reason lots of companies don't like, even tho no coder isn't constantly googling on the job.

I thought about the AI pollution and I don't think it will matter because we already had the exact same problem with forum spam and low information / bad people (=eternal september) type content overwhelming online discourse. I think maybe combatting AI spam will be the least of our problems because humans do plenty of AI-tier spam already. I think the advent of AI may even be positive because people will start to value real human interaction more. It will be like a treasure cove when you find a forum that is highly curated with real people and you can rely on that being the case. Or (and this is just as likely): Online platforms will go towards requiring real human IDs for participants. AI spam would only be used by companies and advertisers.

Maybe eventually there will be a public understanding that only a foolish child would read a reddit frontpage and expect human content. It will all be auto generated to trap people, like a gatcha game.

> Online platforms will go towards requiring real human IDs for participants.

I just don't see how AI won't be used to manipulate these kinds of barriers. Once AI reaches a point where it can truly equal or surpass humans in terms of operating online, how are we going to prevent it from circumventing those barriers? If AI can generate code from a chat prompt, who's to say it can't solve a captcha in the future? And once that happens (because we all know it will at some point), how are we going to be able to differentiate?

real human ids = passports, bank logins

Ok, but the systems that are responsible for creating those IDs are automated, which means that they can be learned and reverse-engineered. There are lots of passports and bank logins floating around that can be used to train AI.

Do you see the problem now?

they are not automated and require real world steps

Passports contain digitally signed X.509 certificates in their chips. No AI can learn to forge a digital signature (we hope).

So you buy them signed at the source.

The only passport you can buy is your own, right? The signing keys are held by national governments. I don't really understand the threat model here.

A government sells you passports without validation of your identity.

The other problem is that AI can be put in charge of phishing operations. Once you devise the correct prompt to get past the filters, it understands what phishing is, and will quite happily write phishing emails for any audience you care to describe. Combine that with an automatic mailer, and you could just fish for IDs (along all the other profitable stuff), and then use those IDs to spam more etc.

There is a market for KYC'd accounts. Someone operating an AI bot farm posing as Real Humans would just buy these.

> I think AI is going to force us on some level to reclaim some of our communication to the physical world, if for no other reason than it's going to be one of the only ways we can truly verify that we're talking to another person.

I've been thinking along these lines a lot recently - it seems as though in every field which AI enters, it causes us to return to some extent to the physical world.

yeah, agree. I think it will be overall a degenerative process.

More and more content that gets created by machine that is of low quality will get in the way of its own future training.

There will eventually be less and less human made content to train from.

Alternatively, we might become more journalistic/academic. Naming and checking your sources will become paramount.

I hope so, but the cynic in me doesn't see this happening because this has long been a problem that isn't going away.

The better that computers get at generating content and behaving in ways that only humans used to be able to is going to make it harder to determine if a source is human or not.

To an extent yes, but this can quickly become overwhelming.

For example, editors and reviewers for academic journals / conferences will likely see a deluge of AI-generated "scientific" papers. Their time is limited, and odds are that more papers with incorrect information will slip through the peer review process.

To be clear, peer review today certainly isn't perfect at catching bad papers. But AI generation has the potential to exaggerate the problem.

For inspiration, here is how the NYTimes deals with anonymous sources:

What we consider before using anonymous sources: How do they know the information?

What’s their motivation for telling us?

Have they proved reliable in the past?

Can we corroborate the information they provide?

Because using anonymous sources puts great strain on our most valuable asset: our readers’ trust, the reporter and at least one editor is required to know the identity of the source. A senior newsroom editor must also approve the use of the information the source provides.

That's already been a problem for some years already:

https://dailysceptic.org/2022/06/08/fake-science-the-threat-...

The sad thing is it doesn't take a ChatGPT level intelligence to beat scientific peer review. Journals routinely publish papers that are completely auto-generated gibberish. A simple generative grammar or template splicer is apparently enough. These are articles that are immediately visible as the work of a program at first glance, they wouldn't make it past even the most amateur blog or student newspapers, yet they surface in their thousands in journals that are supposed to be the epitome of accurate knowledge!

Worse, the journal publishers are doing nothing about it. Their current approach to trying to fix the problem is to try and use the work of random CS academics to make "spam filters" for paper submissions. The more obvious solution of having editors and reviewers who actually read scientific papers before they are published appears to be rejected out of hand.

Reply on news.ycombinator.com