Hacker News Reader

Comment by jdietrich | Hacker News Reader

A Google search for practically any long-tail keywords will reveal that LLMs have already had a very significant impact. DuckDuckGo has suffered even more. Social media is absolutely lousy with AI-powered fraud of varying degrees of sophistication.

It's glib to dismiss safety concerns because we haven't all turned into paperclips yet. LLMs and image gen models are having real effects now.

We're already at a point where AI can generate text and images that will fool a lot of people a lot of the time. For every college-educated young person smugly pointing out that they aren't fooled by an image with six-fingered hands, there are far more people who had marginal media literacy to begin with and are now almost defenceless against a tidal wave of hyper-scaleable deception.

We're already at a point where we're counselling elders to ignore late-night messages from people claiming to be a relative in need of an urgent wire transfer. What defences do we have when an LLM will be able to have a completely fluent, natural-sounding conversation in someone else's voice? I'm not confident that I'd be able to distinguish GPT-4o from a human speaker in the best of circumstances and I'm almost certain that I could be fooled if I'm hurried, distracted, sleep deprived or otherwise impaired.

Regardless of any future impacts on the labour market or any hypothesised X-risks, I think we should be very worried about the immediate risks to trust and social cohesion. An awful lot of people are turning into paranoid weirdos at the moment and I don't particularly blame them, but I can see things getting seriously ugly if we can't abate that trend.

View on HN

> I'm not confident that I'd be able to distinguish GPT-4o from a human speaker in the best of circumstances and I'm almost certain that I could be fooled if I'm hurried, distracted, sleep deprived or otherwise impaired.

Set a memorable verification phrase with your friends and loved ones. That way if you call them out of the blue or from some strange number (and they actually pick up for some reason) and you tell them you need $300 to get you out of trouble they can ask you to say the phrase and they'll know it's you if you respond appropriately.

I've already done that and I'm far less worried about AI fooling me or my family in a scam than I am about corporations and governments using it without caring about the impact of the inevitable mistakes and hallucinations. AI is already being used by judges to decide how long people should go to jail. Parole boards are using it to decide who to keep locked up. Governments are using it to decide which people/buildings to bomb. Insurance companies are using to deny critical health coverage to people. Police are using it to decide who to target and even to write their reports for them.

More and more people are going to get badly screwed over, lose their freedom, or lose their lives because of AI. It'll save time/money for people with more money and power than you or I will ever have though, so there's no fighting it.

"Hey Janelle, what's wrong with Wolfie?"

Your foster parents are dead

Another amazing demo of an AI talking to another AI over a phone line.

Or just ask them to tell them something only you both know (a story from childhood, etc). Reminds me of a book where this sort of thing was common (don't remember the title):

1. something you have

2. something you know

3. something you are

These three things are required for any authz.

For many people it would be better to choose specific personal secrets due to the amount of info online. I'm not a very active social media user, and what little I post tends not to be about me, but from reading 15 year old Facebook posts made by friends of mine you could definitely find at least one example on each of those categories. Hell, I think probably even from old work-related LinkedIn posts.

We had a “long lost aunt” come out of nowhere that got my phone number from a relative who got my number from another relative.

At that point, how can you validate it, as there’s no shared secret? The only thing we had was validating childhood stories. After a preponderance of them, we accepted she was real (she refused to talk on the phone — apparently her voice was damaged).

We eventually met her in real life.

The point is, you can always use these three principles: asking relatives to validate the phone number — something you have — and then the stories — something shared — and finally meeting in real life — something you are.

Oh, you remember those little games that your mom played on facebook/tic tok that asked her "Her favorite", sorry she already trained the AI who she was.

I only say this sort of jokingly. Three out of four of my parents/in laws are questionably literate on the internet. It wouldn't take much of a "me bot" for them to start telling it the stories of our childhood and then that information is out there.

People are and have always been screwed over by modestly equiped humans.

Lincoln already made that observation in the 1850s, "You can fool some of the people all of the time, and all of the people some of the time"

As technology advances those proportions will be boosted. Seems inevitable.

Not sure how much of that has to do with technology or simply a widening gap in people's education we seem to be seeing for a while now.

The way to get around your side channel verification phrase is by introducing an element of stress and urgency: "omg, help, I'm being robbed and they need $300 immediately or they'll hurt me, no time for a passphrase!" can additionally feign memory loss.

Alternatively while it may be difficult to trick you directly, phishing the passphrase from a more naive loved one or bored coworker and then parroting it back to you is also a possibility. 'etc.

Phone scams are no joke and this is getting past the point where regular people can be expected to easily filter them out.

"Hey mom and dad, we need a memorable phrase so AI bots can't call us and pretend to be each other."

I think humankind has managed massive shifts in what and who you could trust several times before.

We went from living in villages where everyone knew each other to living in big cities where almost everyone is a stranger.

We went from photos being relatively reliable evidence to digital photography where anyone can fake almost anything and even the line between faking and improving is blurred.

We went from mass distribution of media being a massive capital expenditure that only big publishers could afford to something that is free and anonymous for everyone.

We went from a tiny number of people in close proximity being able to initiate a conversation with us to being reachable for everyone who could dial a phone number or send an email message.

Each of these transitions caused big problems. None of these problems have ever been completely solved. But each time we found mitigations that limit the impact of any misuse.

I see the current AI wave as yet another step away from trusting superficial appearances to a world that requires more formal authentication protocols.

Passports were introduced long ago but never properly transitioned into the digital world. Using some unsigned PDF allegedly representing a utility bill as proof of address seems questionable as well. And the way in which social security numbers are used for authentication in the US is nothing short of bizarre.

So I think there are some very low hanging fruits in terms of authentication and digital signatures. We have all the tools to deal with the trust issues caused by generative AI. We just have to use them.

During these boundaries people can die. Consider the advent of yellow journalism and the connection with the Spanish-American war 1898: https://en.m.wikipedia.org/wiki/American_propaganda_of_the_S...

No doubt, people die of absolutely everything ever invented and also of not having invented some things.

The best we can ever hope to do is find mitigations as and when problems arise.

Which is why we started saying "whoa, slow down" when it came to some particular artifacts, such as nuclear weapons as to avoid the 'worse than we can imagine' scenario.

Of course this is much more difficult when it comes to software, and very few serious people think the idea of a ever present government monitoring your software would be a better option then reckless AI development.

Outside of the transition to a large city, virtually everything you've mentioned happened in the last 1/2 century. Even the phone was expensive, and not widely in use in under 100 years ago.

That's massive fast change, and we haven't culturally caught up to any of it yet.

Here's another one: We went from in-person story telling to wide distribution of printed materials, sometimes by pseudonymous authors.

This happened from the 15th century onward. By the 19th century more than half the UK population could read and write.

I tend to think the answer is to go back to villages, albeit digital ones. Authentication only enforces that an account is accessed by the correct "user", but particularly in social media many users are bad actors of various stripes. The strongest account authentication in the world doesn't help with that.

So the question, I think, is how do we reclaim trust in a world where every kind of content can be convincingly faked? And I think the answer is by rebuilding trust between users such that we actually have reason to simply trust the users we're interacting with aren't lying to us (and that also goes for building trust in the platforms we use). In my mind, that means a shift to small federated and P2P communication since both of these enable both the users and the operators to build the network around existing real-world relationships. A federation network can still grow large, but it can do so through those relationships rather than giving institutional bad actors as easy of an entrance as anyone else.

But this causes other problems such as the emergence of insular cultural or social cliques imposing implicit preconditions for participation.

Isn't it rather brilliant that you can just ask questions of competent people in some subreddit without first becoming part of that particular social circle?

It could also reintroduce geographical exclusion based on the rather arbitrary birth lottery.

More tech won’t solve it. Areas, either physical or logical, with no or low tech might help.

Just because we haven't yet destroyed the human race through the use of nuclear weapons doesn't mean that it can't or won't happen now that we have the capacity to do so. And I would add that we developed that capacity in less than 50 years of creating the first atomic bomb. We're now living on a knife's edge and at the merge of safeguards which we don't give much thought to on a daily basis because we hope that they won't fail.

That's how I look at where we're going with AI. Plunge along into the new arms race first and build the capacity, then later figure out the treaties and safeguards which we hope will keep our society safe (and by that I don't mean a Skynet-like AI-powered destruction, but the upheaval of our society potentially as impactful as the industrial revolution.)

Humanity will get through it, I'm sure. But I'm not confident it will be without a lot of pain and suffering for a large percentage of people. We also managed to survive 2 world wars in the last century--but it cost the lives of 100 million people.

> Each of these transitions caused big problems. None of these problems have ever been completely solved. But each time we found mitigations that limit the impact of any misuse.

This a problem with all technology. The mitigations are like technical debt but with a difference. You can fix technical debt. Short of societal collapse mitigations persist, the impacts ratchet upward and disproportionately affect people at the margin.

There's an old (not quite joke) that if civilization fell, a large percentage of the population would die of the effects of tooth decay.

Sure, all tech has 'real' effects. It's kinda the definition of tech. But all of these concerns more or less fall into the category of "add it to the list of things you have to watch out for living in the 21st century" - to me, this is nothing crazy (yet)

The nature of this tech itself is probably what is getting most people - it looks, sounds and feels _human_ - it's very relatable and easy for a non-tech person to understand it and thus get creeped out. I'd argue there are _far_ more dangerous technologies out there, but no one notices and / or cares because they don't understand the tech in the first place!

>to me, this is nothing crazy (yet)

The "yet" is carrying a lot of weight in that statement. It is now five years since the launch of GPT-2, three years since the launch of GPT-3 and less than 18 months since the launch of ChatGPT. I cannot think of any technology that has improved so much in such a short space of time.

We might hit an inflection point and see that rate of improvement stall, but we might not; we're not really sure where that point might lie, because there's likely to still be a reasonable amount of low-hanging fruit regarding algorithmic and hardware efficiency. If OpenAI and their peers can maintain a reasonable rate of improvement for just a few more years, then we're looking at a truly transformational technology, something like the internet that will have vast repercussions that we can't begin to predict.

The whole LLM thing might be a nothingburger, but how much are we willing to gamble on that outcome?

If we decide not to gamble on that outcome, what would you do differently than what is being done now? The EU already approved the AI act, so legislation-wise we're already facing the problem.

The EU AI act - like all laws - only matter to those which are required to follow said law.

Yes, but it's really hard to see a technical solution to this problem, short of having locked down hardware that only runs signed government-approved models and giving unlocked hardware only to research centers. Which is a solution that I don't like.

If you get off the internet you'd not even realise these tools exists though. And for the statement that all jobs will be modelled to be true, it'd have to be impacting the real world.

Capabilities aren't the problem, cultural adoption is. Just yesterday I talked to someone who still googles solutions to their Excel table woes. Didn't they know of Copilot?

Maybe they didn't know, maybe none of their colleagues used it, their company didn't pay for it, or maybe all they need is an Excel update.

But I am confident that using Copilot would be faster than clicking through the sludge that are Microsoft Office help pages (third party or not.)

So I think it is correct to fear capabilities, even if the real world impace is still missing. When you invent an airplane, there won't be an airstrip to land on yet. Is it useless, won't it change anything?

I don't see how "failing to use the best tools available" is a relevant problem for this topic, even though it is indeed a problem in other regards.

Copilot in excel is really awful.

If you get away from roads you wouldn't realize engines exist. Also, the internet is (part of) the real world.

Sure and there’s endless AI generated blog spam from “journalists” saying LLMs are amazing and they’re going to replace our jobs etc… but get away from the tech bubble and you’ll see we’re so far away from that. Full self driving when? Autonomous house keepers when? Even self checkout still has to have human help most of the time and didn’t reduce jobs much. Call me a skeptic but HN is way too optimistic about this stuff.

Replacing all jobs except LLM developers? I’ll tell my hairdresser

If we could predict "when", that would make the investment decisions much easier.

But we do have a huge number of examples of jobs disappearing thanks to machines — even the term "computer" used to refer to a job.

More recently and specifically to LLMs, such losses were already being reported around this time last year: https://www.businessinsider.com/jobs-lost-in-may-because-of-...

In a world where openAI exists, it really does require an almost breathtaking lack of imagination to be a skeptic.

Or you’ve been around the block long enough to recognize hype and know when your imagination may be wrong. Imagination isn’t infallible.

Right, that entire internet think was complete hype, didn't go anywhere. BTW, can you fax me the menu for today?

And that motorized auto transport, it never went anywhere, it required roads. I mean, who would ever think we'd cover a huge portion of our land in these straight lines. Now, don't mind me, I'm going to go saddle up the horse and hope I don't catch dysentery on the way into town.

I don't think anybody's denying that revolutions happen. It's just that the number of technologies that actually turned out to be revolutionary are dwarfed by the number of things that looked revolutionary and then weren't. Remember when every television was definitely going to be using glasses-free 3D? People have actually built flying cars and robot butlers, yet the Jetsons is still largely wishful thinking. The Kinect actually shipped, yet today we play games mostly with handheld controllers. AI probably has at least some substance, but there's a non-zero amount of hype too. I don't think either extreme of outcome is a foregone conclusion.

Is it even possible to "get off the internet" without also leaving civilisation in general at this point?

> it'd have to be impacting the real world

By writing business plans? Getting lawyers punished because they didn't realise that "passes bar exam" isn't the same as "can be relied on for citations"? By defrauding people with synthesised conversations using stolen voices? By automating and personalising propaganda?

Or does it only count when it's guiding a robot that's not merely a tech demo?

I’ll be worried about jobs being removed entirely by LLMs when I see something outside of the tech bubble genuinely having been removed by one - has there been any real cases of this? It seems like hyperbole. Most people in the world don’t even know this exists. Comparing it to the internet is insane, based off of its status as a highly advanced auto complete.

> has there been any real cases of this?

Apparently so: https://www.businessinsider.com/jobs-lost-in-may-because-of-...

Note that this article is about a year old now.

> Comparing it to the internet is insane, based off of its status as a highly advanced auto complete.

(1) I was quoting you.

(2) Don't you get some cognitive dissonance dismissing it in those terms, at this point?

"Fancy auto complete" was valid for half the models before InstructGPT, as that's all the early models were even trying to be… but now? The phrase doesn't fit so well when it's multimodal and can describe what it's seeing or hearing and create new images and respond with speech, all as a single unified model, any more than dismissing a bee brain as "just chemistry" or a human as "just an animal".

800 million dollar studio expansion halted - https://www.theguardian.com/technology/2024/feb/23/tyler-per...

Thank god! Enough Medea, already! I chalk this up as a win for humanity.

Sure, but think about all of the jobs that won't exist because this studio isn't being expanded, well beyond just whatever shows stop being produced. Construction, manufacturing, etc.

Edit: Also this doesn't mean less medea, just less actual humans getting paid to make medea or work adjacent jobs

Not like there's nothing else to construct.

Maybe it's time to construct some (high[er] density) housing where people want to live? No? Okay, then maybe next decade ... but then let's construct transport for them so they can get to work, how about some new subway lines? Ah, okay, not that either.

Then I guess the only thing remains to construct is all the factories that will be built as companies decouple from China.

"By 2005 or so, it will become clear that the Internet’s impact on the economy has been no greater than the fax machine’s."

~ Paul Krugman, winner of the 2008 Nobel Memorial Prize in Economic Sciences

HN comments, too. Long, grammatically perfect comments that sound hollow and a bit lengthy are everywhere now.

It's still early, and I don't see much in corporate communications, for instance, but it will be quite the change.

>Long, grammatically perfect comments that sound hollow and a bit lengthy

It's worse than I thought. They've already managed to mimick the median HN user perfectly!

Yes. The old heuristics of if something is generated by grammar and sentence structure don't work as well anymore. The thing that fucks me up the most about it is that I now constantly have to be uncertain about whether something is human or not. Of course, you've always had to be careful about misinformation on the internet, but this raises the scalability of false, hollow, and harmful output to new levels. Especially if it's a topic I'm trying to learn about by reading random articles (or comments), there isn't much of a frame of reference to what's good info and what's hallucinated garbage.

I fear that at some point the anonymity that made the internet great in the first place will be destroyed by this.

To be fair that was already the case for me before AI, Right at that time that companies, individual and governments found out that they could write subvert ads in the form of comments posts and 'organic' and they started to flood reddit, discord, etc.

The dead internet theory started to look more real with time, AI spam is just scaling it up.

I have a strange problem: I have always written in a manner that now leads people to think I am a LLM.

It has been so bad, I even considered injecting misspelling and incorrect grammar and bad punctuation into my prose to prove my words are mine.

I feel you, people who liked the word "delve" will have to stop using it.

I tried to make ChatGPT generate a counterpoint for that but it turns out you're right.

No problem, I'm here to keep the language sophistication level low.

I take care of the etiquette and try to do my best to keep it low.

We need one who's doing the dirty work of not discussing.

Many business doesn't want to send their data to a third party such as OpenAI, so until locally run LLM becomes wildly available in businesses.

as the meme goes "always has been"

i remember seeing the change when GPT-2 was announced

I'm a non native English speaker. Edge new feature of automatically improving my text is a God send. Unfortunately it is blocked at work.

We’ve reached a stage, where it would be advisable to not release recent photos of yourself, nor any video with sound clips to public, unless you want an AI fake instaperson of yourself starting to reach out to member of your externally visible social network, asking for money, emergency help, etc.

I guess we need to have an AI secretary to take in all phonecalls from now on (spam folder will become a lot more interesting with celebrity phone calls, your dead relative phoning you etc)

Hopefully, we will soon enter the stage where nobody believes anything they see anymore. Then, you no longer have to be afraid of being misinterpreted, because nobody is listening anymore anyway. Great time to be alive!

I don't know why anyone thinks this will happen. You can obviously write anything you want (we have an entire realm of works in this area that everyone knows about, fiction) and yet huge amounts of people believe passed around stories either from bad or faked media sources or entirely unsourced.

I'm not saying either you or the parent commenter is right or wrong, but fiction in books and movies are clearly fiction and we consume it as such. You are right that some people have been making up fake stories and others (the more naive) have been quick to believe in those false stories. The difference now is that it's not just text invented and written by a human, which takes time and dedication. Now it's done in a second. On top of that it's easy to enhance the text with realistic photos, audio and video. It becomes much more convincing. And this material is created in a few seconds or minutes.

It's hard to know what to believe if you get a phone call with the voice of your child or colleague, and your "child"/"colleague" replies within milliseconds in a convincing way.

I agree it's fundamentally different in application which I think will have a large impact (just like targeted advertising with optimisation vs billboards), but my point is that given people know you can just write anything and yet misinformation is abound - I don't see how knowing that you can fake any picture or video or sound leading to a situation where everyone just stops believing them.

I think unfortunately it will massively lower the trust of actual real videos and images, because someone can dismiss them with little thought.

Luckily there’s a “solution” to that: Just don’t use the internet for dialogue anymore.

As someone that grew up with late-90’s internet culture and has seen all the pros and cons and changes over the decades, I find myself using the internet less and less for dialogue with people. And I’m spending more time in nature and saying hi to strangers in reality.

I’m still worries about the impact this will have on a lot of people’s ability to reason however. “Just” Tik Tok and apps like it has already had devastating results on certain demographics.

Statistically, more and more people spend time online and on phone. I'm not sure if we still reached the peak in terms of internet usage.

That bit "... there's a "solution"" - does it keep working in societies where there are mega corps pushing billions into developing engaging, compelling and interesting AI companions?

That's why I put it in quotation marks because it is a solution that will remain available, simply because the planet is really big and there'll always be places on the fringes. But it doesn't really solve the problem for society at large, it only solves it for an individual. But sometimes individuals showing other ways of living helps the rest of society see that there's choices where they previously thought there were none.

Be glib, but that is one way for society to bring privacy back-and with it shared respect. I think of it as the “oh everyone has an anus” moment. We all know everyone has one and it doesn’t need to be dragged out in polite company.

I'm not sure if people work like that — many of us have, as far as I can tell for millennia and despite sometimes quite severe punishments for doing so, been massive gossips.

We're doing FSB's work for them. Or PLA Unit 61398's (or their comrades). Or Bureau 121's.

Brave New World indeed.

What you see will be custom tailored to what you believe, and your loyalty will be won. Do what the AI says and your life will be better. It already knows you better than you know yourself. Maybe you're one of those holdouts who put off a smartphone until life became untenable wihout it. Life will be even more untenable without your AI personal assistant/friend/broker/coach/therapist/teacher/girlfriend to navigate your life for you.

I'm paranoid enough that I now modulate my voice and speak differently when answering an unknown phone call just in case they are recording and building a model to call back a loved one later. If they do get a call, they will be like, "why are you talking like that?"

I can only imagine "Hello, this is Guy Incognito..."

Better yet, don’t answer unknown phone calls.

> unknown phone calls

This is my biggest gripe against the telecom industry. Calls pretending to be from someone else.

For every single call, someone somewhere must know at least the next link in the chain to connect a call. Keep following the chain until you find someone who either through malice or by looking the other way allows someone to spoof someone else's number AND remove their ability to send the current link in the chain (or anyone) messages. (Ideally also send them to prison if they are in the same country.) It shouldn't be that hard, right?

Companies have complex telecoms but generally want the outside as one company number. Solution, the sender send a packet with the number they should get perceived as. Everyone sends this on. Everyone "looks the other way" by design haha

So what, gate that feature behind a check that you can only set an outgoing caller ID belonging to a number range that you own.

The technology to build trustable caller ID has existed for a long time, the problem is no one wants to be the one forcing telcos all over the world to upgrade their sometimes many decades old systems.

But why not just make up a secret word to use with your beloved ones in critical situations. In case of ..., one needs to know that secret. Otherwise, FAKE! Gotcha!

Secrets are either written down somewhere and end up on the Internet, or forgotten.

It doesn't have to be the unspeakable, but rather can be the name of the first pet or something others just can't guess on the first time.

The problem here is you're assuming your family members aren't idiots, this is your first mistake.

Chances are they've already shoved some app on their phone that's voice to txting everything they say and sending off somewhere (well lower chance if they have an iphone).

Modern life is data/information security and humans are insanely bad at it.

By chance, they are noobs but not idiots, because they ask me on everything - they don't need Google, I know everything hahah

I don't think it's a problem to find a word or a sentence or a story - whatever - that's commonly used by everyone on daily basis but in different context. That's not a problem by itself :) try it

For the idiots, it is still possible to find a word. They may be idiots, but still, they work and live on their own. They coming along in life. So, it's up to the smarter one to find a no-brainer solution.

I am confident and believe nothing and no one is stupid enough not to be able to adapt to something. Even if it's me, who'll need to adapt to members with less brain.

I think for most people it's far too late, as there exists at least something on the internet and that something is sufficient - photos can be aged virtually and a single photo is enough, voice doesn't change much and you need only a tiny sample, etc.

And that's the case even if you've never ever posted anything on your social media - it could be family&friends, or employer, or if you're ever been in a public-facing job position that has ever done any community outreach, or ever done a public performance with your music or another hobby, or if you've ever walked past a news crew asking questions to bystanders of some event, or if you've ever participated in some contests or competitions or sports leagues, etc, all of that is generally findable in various archives.

> photos can be aged virtually and a single photo is enough

I'm sure AI-based ageing can do a good enough job to convince many people that a fake image of someone they haven't seen for years is an older version of the person they remember; but how often would it succeed in ageing an old photo in such a way that it looks like a person I have seen recently and therefore have knowledge rather than guesses about exactly what the years have changed about them?

(Not a rhetorical question to disagree with you, I genuinely have no idea if ageing is predictable enough for a high % result or if it would only fool people with poor visual memory and/or who haven't seen the person in over a decade.)

I feel like even ignoring the big unknowns (at what age, if any, will a person start going bald, or choose to grow a beard or to die their hair, or get a scar on their face, etc.) there must be a lot of more subtle but still important aspects from skin tone to makeup style to hair to...

I've looked up photos of some school classmates that I haven't seen since we were teens (a couple of decades ago), and while nearly all of them I think "ah yes I can still recognise them", I don't feel I would have accurately guessed how they would look now from my memories of how they used to look. Even looking at old photos of family members I see regularly still to this day, even for example comparing old photos of me and old photos of my siblings, it's surprising how hard it would be for a human to predict the exact course of ageing - and my instinct is that this is more down to randomness that can't be predicted than down to precise logic that an AI could learn to predict rather than guess at. But I could be wrong.

Maybe it's Europeans posting this kind of stuff where they have much stronger privacy laws, but if you're in the US this is all wishful thinking.

Do you shop in large corporate stores and use credit cards? Do you go out in public in transportation registered to you?

If yes, then images and habits of yours are being stored in databases and sold to data brokers.

And you're not even including every single one of your family members that use internet connected devices/apps that are sucking up all the data they can.

I was just asking about the ability of photo aging software, not commenting about privacy at all. Though yes, I am thankfully in Europe (but there are recent photos of me online).

But don't disagree with you - in a different comment that was about privacy, I (despite living under GDPR) suggested that for offline verification with known people it's better to choose secrets that definitely haven't been shared online/anywhere rather than just choosing random true facts and assuming they couldn't have been found out by hackers: https://news.ycombinator.com/item?id=40353820

> I guess we need to have an AI secretary to take in all phonecalls

Why not an AI assistant in the browser to fend all the adversarial manipulation and spam AIs on the web? Going online without your AI assistant would be like venturing without a mask during COVID

I foresee a cat-and-mouse game, AIs for manipulation vs AIs for protection one upping each other. It will be like immune system vs viruses.

> Social media is absolutely lousy with AI-powered fraud of varying degrees of sophistication.

has been for years mon ami. i remember when they started talking about GPT-2 here, and then seeing a sea-change in places like reddit and quora

quite visible on HN, esp. in certain threads like those involving brands that market heavily, or discussions of particular countries and politics.

People were already killing each other for thousands of years so introducing tanks was no big deal, I guess. To say nothing of nuclear weapons.

What does abating that trend look like? Most AI safety proposals I hear fall into the categories of a) we need to stop developing this technology or b) we need laws that entrench the richest and most powerful organizations in the world as the sole proprietors of this technology. Neighther of those actually sound better than people being paranoid weirdos about trusting text/video/voice. I think that's kinda where we need to be as a culture: these things are not trustworthy, they were only ever good as a rough heuristic, and now that ship has sailed. We have just finished a transition to treating the digital world as part of our "real" world, but it's time to step that back. Using the internet to interact with known trusted parties will still work fine, provided that some authentication can be shared out-of-band offline. Meeting people and discovering businesses and such? There will be more fakes and scams than real opportunities by orders of magnitude, and as technology progresses our filtering will only get worse. We need to roll back to "don't trust anything online, don't share your identity or payment information online" outside of, as mentioned, out-of-band verified parties. You can still message your friends and family, do online banking and commerce, but you can't initiate a relationship with a person or business online without some kind of trusted recommendation.

>What does abating that trend look like?

I don't think anyone has a good answer to that question, which is the problem in a nutshell. Job one is to start investing seriously in finding possible answers.

>We need to roll back to "don't trust anything online, don't share your identity or payment information online"

That's easy to say, but it's a trillion-dollar decision. Alphabet and Meta are both worthless in that scenario, because ~all of their revenue comes from connecting unfamiliar sellers with buyers. Amazon is at existential risk. The collapse of Alibaba would have a devastating impact on Chinese exporters, with massive consequent geopolitical risks. Rolling back to the internet of old means rolling back on many years worth of productivity and GDP growth.

> because ~all of their revenue comes from connecting unfamiliar sellers with buyers

Well that's exactly the sort of service that will be extremely valuable in a post-trust internet. They can develop authentication solutions that cut down on fraud at the cost of anonymity.

“Extremely valuable” is another way of saying “extremely costly”.

Point a) is just point b) in disguise. You're just swapping companies for governments.

This tech is dangerous, and I'm currently of the opinion that its uses for malicious purposes are far better and more significant than LLM's replacing anyone's jobs. The bullshit asymmetry principle is very incredibly significant for covert ops and asymmetric warfare, and generating convincing misinformation has become basically free overnight.

Trust is more complex then we take credit for.

Even when it comes to people like our parents, there are things we would trust them to do, and things that we would not trust them to do. But what happens when you have zero trusted elements in a category?

At the end of the day, the digital world is the real world, not some seperate place 'outside the environment'. Trying to treat digital like it doesn't exist puts you in a dangerous place to be deceived. For example if you're looking for XYZ and you manage to leak this into the digital world, said digital world may manipulate your trusted friends via ads, articles, the social media posts they see on what they think about XYZ before you ask them.

>Regardless of any future impacts on the labour market or any hypothesised X-risks

Discovering an asteroid full of gold, with as much gold as half the earth to put a modest number, would have huge impact to the labour market. Anything conductive like copper, silver, mining jobs would all go away. Also housing would be obsolete as we would all live in golden houses. A huge impact to the housing market, yet it doesn't seem such a bad thing to me.

>We're already at a point where we're counselling elders to ignore late-night messages from people claiming to be a relative in need of an urgent wire transfer.

Anyone can prove their identity, or identities, over the wire, wire-fully or wire-lessly, anything you like. When i did go to university, i was the only one attending the cryptography class, no one else showed up for a boring class like this. I wrote a story about the Electrona Corp in my blog.

What i say to people for at least 2 years now, is that "Remember when governments were not just some cryptographic algorithms?" Yeah, that's gonna change. Cryptography is here to stay, it is not as dead as people think and it's gonna make a huge blast.

> Discovering an asteroid full of gold, with as much gold as half the earth to put a modest number, would have huge impact

All this would do is crash the gold price. Also note that all the gold at our disposal right now (worldwide) basically fits into a cube with 20m edges (its not as much as you might think).

Gold is not suitable to replace steel as building material (because it has much lower strength and hardness), nor copper/aluminium as conductor (it's a worse conductor than copper and much worse in conductivity/weigth than aluminium). The main technical application short term would be gold plated electrical contacts on every plug and little else...

> The main technical application short term would be gold plated electrical contacts on every plug and little else...

.. And gold teeth and grillz.

Regarding gold, i like this infographic [1], but my favorite from this channel is wolf population by country. Point being, that gold is shiny and beautiful, and it will be used even when it is not appropriate solution to the problem, just because it is shiny.

I didn't know that copper is a better conductor than gold. Surprised by that.

[1] https://www.youtube.com/watch?v=E2Gd8CRG0cc

> What i say to people for at least 2 years now, is that "Remember when governments were not just some cryptographic algorithms?" Yeah, that's gonna change. Cryptography is here to stay, it is not as dead as people think and it's gonna make a huge blast.

The thing about cryptography and government is that it's easy to imagine for a great technology to be adapted on the governmental level because of its greatness. But it is another thing to actually implement it. We live in a bubble, where almost anyone knows about cryptographic hashes and RSA, but for most of the people it is not the case.

Another thing is that political actors are tending to try to concentrate power in their own hands. No way they will delegate a decision making to any form of algorithm — being cryptographic or not.

As soon as mimicking voices, text messages, human faces becomes a serious problem, like this case in UK [1], then citizens will demand a solution to that problem. I don't personally know how prevalent problems like that are as of today, but given the current trajectory of A.I. models which become smaller, cheaper and better all the time, soon everyone on the planet will be able to mimic every voice, every face and every handwritten signature of anyone else.

As soon as this becomes a problem, then it might start bottom-up, citizens to government officials, rather than top to bottom, from president to government departments. Then governments will be forced to formalize identity solutions based on cryptography. See also this case in Germany [2].

One example like that, is bankruptcy laws in China. China didn't have any law regarding to bankruptcy till 2007. For a communist country, or rather not totally capitalist country like China, bankruptcy is not an important subject. When some people stop being profitable, they will keep working because they like to work and they contribute to the great nation of China. That doesn't make any sense of course, so their government was forced to implement some bankruptcy laws.

[1]https://www.wsj.com/articles/fraudsters-use-ai-to-mimic-ceos... [2]https://news.ycombinator.com/item?id=39866056

A lot of these are non-AI problems. People trying to defraud the elderly need to be taken out back and shot, that’s not an AI issue.

Right, I'll just get right on a plane and travel to whereverthefuckville overseas and ask for permission to face blast the scammers. The same scammers that are donating a lot of money to their local (probably very poor) law enforcement to keep their criminal enterprise quite. This will go well.

> I'm not confident that I'd be able to distinguish GPT 4o from a human speaker

Probably why it's not released yet. It's unsafe for phishing.

I think people are dismissive for a few reasons.

- It helps them sleep at night if their creation doesn't put millions of people out of work.

- Fear of regulation

> What defences do we have when an LLM will be able to have a completely fluent, natural-sounding conversation in someone else's voice?

The world learnt to deal with Nigerian Prince emails and nobody is falling to those anymore. Nothing was changed - no new laws or regulations needed.

Phishing calls have been going on without an AI for decades.

You can be skeptical and call back. If you know your friends or family you should be able to find an alternative way to get in touch always without too much effort in the modern connected world.

Just recently a gang in Spain was arrested for "son in trouble" scam. No AI used. Most of the parents are not fooled in this.

https://www.bbc.com/news/world-europe-68931214

The AI might have some marginal impact, but it does not matter in the big picture of scams. While it is worrisome, it is not a true safety concern.

Reply on news.ycombinator.com