Hacker News Reader

Comment by mgraczyk | Hacker News Reader

Really concerning to me that people find the luddite argument so persuasive and that it gets so much play in the press. The crux of the argument from the outside "ethicists" quoted in the article is something like.

"This new piece of technology might be dangerous and we don't fully understand it, so we should not poke at it or allow people to study it."

Maybe there's just something about my personality that is deeply at odds with this sentiment, but it's also about the lack of testable predictions coming from people like this. Their position could be taken about literally anything with the same logical justification. It's a political and emotional stance masquerading as a technical or scientific process.

View on HN

Here is one prediction: Because of language models, the amount of fake news online will increase by an order of magnitude (at least) before this decade ends. And there is no interpretation of that development as anything else but a net negative.

Another more broad prediction: In a decade, the overall influence of language models on our society will be universally seen as a net negative.

What if that's the push that brings people out into the physical world where they don't have to deal with all this crap online.

Slightly less optimistic, but perhaps more realistic thought: what if that's the push that makes people validate their (internet) sources? Seems like it might become clear that random web pages are just automatically generated content. If you really want to learn something credible at all, you'll really have to be more specific about your sources than "the internet".

Yeah, that was gonna be my contrarian HN hot-take too. Basically if it becomes really obvious some day that basically all online news publication is noise written by computers, maybe people will stop actually trusting it so much?

Fake news and AI generated news are kinda all over the place for a good amount of time. It's faster and cheaper to have AI write news from a press release.

My prediction is that in the next 10y we will really struggle to determine between fake-people and real-human. There will be an explosion of fake-identities posting more and more human-like.

But I'm not Nostradamus so I could be very very off here.

You’re probably right but it will be an arms race with ever more sophisticated mitigation techniques being deployed to filter.

I’d say Neil Stephenson has a pretty good take on what this might look like in his recent book: Fall, where in everyone has a “feed” and those that or more savvy/wealthy have better editors (AI, etc) of their feed.

It's all about having the right tools, but I wonder how long can we "beat the machine" :)

It depends. On forums like this it would basically take a machine that would pass the Turing test in order not to be seen as an AI in any "meaningful" conversation that it might join (so, not just a comment posted as a reply here and there).

And even if the powers that be manage to get those future AI bots to post stuff that will very much resemble what we now post in here, it is my belief that the uncanny valley will be, in the end, impossible to pass (in fact that's one of the main motifs of many of Asimov's books when it comes to robots).

I'm not particularly convinced by the Dead Internet Theory [1] as of 2022, in the sense that it is completely true right now. But I am convinced it is building around us, and even now, the correct frame for the question isn't whether it is true, but how true it is. There is too many entities with too much reason to build it for it not to be happening. And the nature of it is such that it doesn't need to be one entity doing it for one unified reason; dozens, hundreds can all be participating & fighting with each other on different levels and the sum total of all that is to put it together that much more quickly.

You know, the PGP web of trust idea may yet take off all these decades later, not because we need a web of trust to send 100.0000000% safely encrypted messages to each other to protect from governments, but because we need a web of trust just to know who the real humans are.

[1]: https://forum.agoraroad.com/index.php?threads/dead-internet-...

Curious to what degree and how you think web of trust idea could help here? Assume you could use it to prove whether an article was signed or not by a real person. I think this would solve the problem of articles being published with a false author attribution. However, it would not work to prevent actual people from publishing AI written articles using their own identity. It would also not (directly) do anything to establish if the facts in an article are correct or not.

Specifically, a web of trust designed for exactly that assertion: This person is a real human. Signatures here serve to assert what identity something came from.

There would be some of the usual web of trust problems, e.g., trying to explain to Joe Q. Public that you only sign for people you know, beyond doubt, are human. Preferably in person. Many other problems, too.

I guess you could say, my thought here isn't that this would solve the problems. The problems at this point are somewhat well known. What has been missing is any benefit significant enough to motivate us to get past those problems. If there is, it's obviously many years away. Wouldn't exactly suggest building a startup around this idea right now, if you get my drift. We still need to go through a phase of the problem getting larger before we even get to the phase where people start to realize this is a problem and start demanding that people online prove they are actually people, and goodness knows "I'm a human" is merely the lowest of low bars itself, not the solution to all trust problems.

I love the idea but hate that I'm so cynical about the likely outcome.

I have already generated fake news-esque things with gpt-3 to send to friends.

A lot of the outputs look incredibly genuine. We live in interesting times.

Cost per word of fake news is already very low though, and humans are much better at tailoring it to the appropriate audience and agenda (and not just restating stuff that's already out there that might actually be true)

GPT type models are much better suited to low effort blogspam, and whilst that's not a good thing, they produce better blogspam than existing blogspamming techniques. I think we underestimate how bad the Internet already is, and at worst text generated by AI is simply going to reflect that.

It's about being 1st to publish it. It is mainly use during live press conferences. No human can snap a photo (at least), write a short update in a dedicated article, and so on... all in 1s.

Been there (as an independent press member years ago), simply you cannot beat that.

First to publish matters, but GPT-3 is neither necessary nor sufficient to achieve that. If you're producing fake news related to a press conference, speed of content generation is entirely unimportant because you don't have to wait for the press conference to start to write the fake article/update/tweet. If you care about fidelity to the message of the press conference, I don't see many situations in which a human who has anticipated the likely message(s) of the conference and therefore has pre-drafted paragraphs about "striking a conciliatory tone" and "confirmed the reversal of the policy" ready to select and combine with a choice quote or two as soon as they're uttered isn't significantly better (and as quick as) GPT-type models prompted by the speech-to-text of the press conference. Sure, more reliable publications will want to stick more validation and at least wait for the conference to finish before summarising its message steps in the way, but those apply to bots as well as journalists (or not, for the publications that prioritise being first over being right).

You have a solid point, but I wasn't talking abut summarizing or excerpt from a press release (those are anyway handed as press-kits before with all NDA agreements and so on).

Real human journalists have a delay of about 1m before making a short tweet. Funny (or not), something similar was in the "live update" article page in less than 10s. Including photo(s). I was on quite a lot of tech-conferences/live events and earned a decent living then as an independent tech journalist (but then I got bored and really it was a 1-man-show).

Another personal observation (from field), that was not happening prior to 2010-2012, the years we all got Siri, Cortana..

You can make the dots and dashes.

Personally, I've never understood why first to publish matters? As far as I can tell, the only people who care are other journalists, who seem to think that any story about a breaking news item MUST credit the person who wrote about it first (see: ESPN's NBA and NFL "insiders").

The unpersuasive argument is "you (second-to-publish-person) copied my stupendous idea to get your derivative result".

If people are looking for news on the topic, they will FB or tweet you first. That has a snowball effect.

That has already happened. manimino linked me to this great page on another thread on HN a few days ago: https://cookingflavr.com/should-you-feed-orioles-all-summer/ But consider that a particularly easy to detect version of the problem. Click some of the other links on that site and have a look. I especially suggest you click something on a topic you know nothing about.

I've been hitting these sites in searches accidentally more and more over the past few months. Goodness help you if you don't realize it's totally fake; some of what I've seen is dangerous, like, bad electrical advice being blithely generated by whichever exact transformer variant is spewing that stuff.

There is an economic incentive to detect machine generated output and curate trusted sites since the feedback loop of training models on an unfiltered internet of mostly generated junk output will eventually lead the models into converging on a useless degenerate state.

Useless Degenerate State... I think I just found the name of my new band!

But here it is in our field of view. This model has very limited look ahead. I don't think we can rely on that in the future.

People don't get their news from random AI-generated blogs.

The actually bad consequence is SEO spam of high quality. You can now generate a hundred articles a minute in any topic.

We are already seeing a lot of fucked up SEO spam rising to the top these days. IMO it might actually start picking at Google's market share because prior to this the product actually seemed infallible to the average layman.

I would not worry about that too much. There are already models that can predict if its transformer generated or not. At the same time google started penalizing transformer generated text; https://youtu.be/EZWx05O6Hss

It doesn't seem like there's any real guarantee that the 'generated text detectors' will outpace the 'generated text generators'

What is the issue with fake news?

Language models can now pass as human in many situations but there are already billions of humans capable of writing fake news, this isn't a new capability.

We have already created mechanisms for deciding which voices to trust and no matter how good language models get they will not be able to prevent you from visiting economist.com

I think the issue could be volume (and also that the vast majority of humans aren't actively exercising their ability to write fake news at present). Also that language models might be far more convincing.

> there are already billions of humans capable of writing fake news

You have to pay them, and most of them are not very good at writing. Even with a big budget, you get a limited number of good articles per day.

If you can make writing fake news 100x cheaper, and then just throw everything at social networks and let people sort out the most viral stuff, that can change the game.

Also, computers can be faster. If something new happens today and hundred articles are written about it, a computer can quickly process them and generate hundred more articles on the same topic, than a group of humans would. (Many humans can do the writing in parallel, but each of them needs to read individually the things they want to react to.)

The vast majority of generators of fake news today are from Content Mills, Copy Paste Mills and general SEO spammers. Political misinformation is the other big generator. The economics of it and not "ethical" gate keeping is what will affect their output. Realistically, normal people don't have the ability to coordinate hordes of proxied IPs to attack social networks requiring account sign ups and custom feeds.

The value of exercising critical thinking, checking trusted curated sources, information hygiene, recognizing and avoiding manipulation tactics in news and ads will have to go up. The internet is already almost entirely unreliable without requiring any fancy AI. The listed skills will be necessary regardless any increase of politically manipulative content, advertisements or product misrepresentations.

Rich people often try to purchase political influence and respect for their opinions. Historically, this has had very mixed results, but you can't deny that it has sometimes worked. The problem is that this does not have a market mechanism. If the rich person has funds from another source, economics can go out the window, and usually does.

> that can change the game.

I don't think people, organizations, and government, pushing false narratives, is some new game. I think it's a game that people are dangerously unaware that they're already playing. Destroying the trust of content on the internet, resulting in having people be more diligent about what they believe, is, I think, almost certainly a net positive.

But, as a counter argument to myself, people are lazy and will, instead, just go to a news source that they "trust", and listen without ever questioning.

Either way, I don't see the fake news, itself, as being anything but a fleeting problem for society. The problem will continue to be people, organizations, and governments taking advantage of laziness.

For an example, imagine Dall-E 2 was released for political use. For a few weeks, we would have a flood of fakes of every politician doing every imaginable act. Society would very quickly believe none, rather than believing all.

> Destroying the trust of content on the internet, resulting in having people be more diligent about what they believe, is, I think, almost certainly a net positive.

That didn't seem to be how it worked with the election-theft narrative. People kept believing garbage from discredited sources.

Yes, because the election-theft narrative was coming from sources they initially trusted (the President of the United States). That is not the same thing as trusting the hundreds of fake news articles that GPT-3 will be able to generate a second.

Agree, this is the same as if IEEE suddenly announced a breakthrough in cold fusion. A large chunk of even the highly educated and technical audience would tend to believe it, or at least give it the benefit of the doubt, simply because of the cachet and reputation IEEE itself has built up. At least until those are exhausted.

GPT-3 can't create a second IEEE so it's not an issue to be worried about.

> Society would very quickly believe none, rather than believing all.

Authoritarians benefit from a society-wide lack of trust in there being any sort of consensus/objective truth. When people don't know what to believe, they either turn to conspiracy theories that claim to offer a peek behind the curtain, or more likely just tune out and hope that their favorite strongman can simplify the chaos.

There's been speculation on Twitter that the conflicting narratives from Russia itself during the invasion aren't really a political problem, since confusion makes it more difficult for the public to rally behind a counter-narrative.

We both agree that language models allow for the quick creation of large amounts of content.

There already exists far more content than anybody can read. We have developed mechanisms for filtering out content which isn't worth our time, and language models don't have any special ability to force you to read their creations.

And even if content can be placed somewhere you will read it: no matter what I put into this box you will not suddenly be susceptible to believing the unbelievable no matter whether I wrote it or whether a language model wrote it.

Is it unbelievable that pressing a bar will bring you long term happiness? Surely it is, but primates will press the heroin bar. This has happened before with humans as well. You cannot simply dismiss the effectiveness of well - targeted propaganda.

Your argument holds true in theory but does not always work in practice.

The issue many people have with fake news is that it's a tool that can sway public opinion without any basis on facts. I'm not sure, by your response, if you find that to be problematic or not.

I think we've recently found that people haven't decided which voices to trust and can be led to believe things placed in front of them. Paired with the ability to spread that information - there is significant impact on society.

That's the reason some people have issues with fake news, from my experience.

Also, getting a computer to do something will always scale several orders of magnitude more than having billions of people do it.

> I think we've recently found that people haven't decided which voices to trust and can be led to believe things placed in front of them. Paired with the ability to spread that information - there is significant impact on society.

> That's the reason some people have issues with fake news, from my experience.

From my experience, most people who believe Fake News is a significant problem are people who dislike WHOM some have chosen to trust.

Few see fake news coming from their preferred media sources, or supporting their preferred narrative, as highly concerning, and instead usually treat it as benign (human errors happen etc etc).

However, when it's coming from their political adversaries, or from sources they dislike, it suddenly is presented as some huge issue.

In reality, it is all the same, and people are decently good at filtering it out when it contradicts what they want to believe, and very bad at filtering it out when it agrees with them.

For an example of fake news from outlets like the NYT, the most egregious recent one has been the dismissal of the Hunter Biden laptop, calling it a Russian hoax, calling it "fake news", listing intelligence agencies (of all people) as proof of this, getting Twitter and Facebook to outright ban the New York Post article on it - when in fact everything in their article was 100% true.

How many people (especially Dem voters) have taken this story as "dangerous spread of Fake News by the NYT, Washington Post etc"?

That's quite the false equivalence. A big part of the issue is quantity. You cherry-picked one example for the NYT, but for Fox News you could find examples on a daily if not hourly basis.

I'm not claiming all news sources are equally trustworthy - obviously the NYT is a better news source than Fox, and both are better than "my primary school friend of Facebook".

My point was that we all have blindspots when the news sources we have chosen to trust are feeding us false information. Also, the problem isn't that people don't choose a news source to trust, it is that they choose a bad news source to trust, and this can happen for very different reasons.

I kind of agree - a lot of "fake" news believers seems to be actively seeking for contrarian views purely for the sake of it, with indoctrination as an incentive offered for labor of reading, rather than harm unto themselves. In that sense, the factual accuracy - the "fake" notion, don't seem to be the point, and the volume of text that NN generators enable can be less of an issue.

Alternative hypothesis for which I have at least as much evidence:

Because of large language models, detecting Fake News becomes trivial and cheap. Building and doing inference on language models is too expensive for most attackers, so they give up. Only well financed state actors are capable of disseminating fake news, and they are no better at it than they are today because content generation is not a bottleneck.

"Because of language models, the amount of fake news online will increase by an order of magnitude (at least) before this decade ends. And there is no interpretation of that development as anything else but a net negative."

That's not at all clear. You're assuming people will continue to give credence to random stuff they read. But once fake AI-generated content is common, people will surely become less trusting. The end result could easily be that fewer people than before believe fake news is real. Presumably, fewer people will believe real news too, but the result could still be net positive.

Bogus content has been available in abundance online for a while and people often are credulous of it still.

But still, you have to assume that the cost of creating fake news is the primary limit to it's appearance in front of people to claim AI will seriously have an impact and that's not at all obvious.

I keep seeing this prediction, but have yet to see a convincing argument as to how this content is supposed to circumvent existing trust networks.

News outlets are nothing without a track record. People trust names they recognize. You can spin up as many CMS instances, domains, and social media profiles for fake news as you want, but without a history shared with its core audience, all the language models in the world aren't going to convince anyone but the most credulous when the content is coming from unfamiliar sources.

Most people now get their news from downstream social media, not from the newsroom. You don't have to sway opinion with fake articles, just a sea of astroturf that supports some opinion and creates the illusion of a crowd, which has traditionally been the main signal for credibility of an idea.

Our grand-grand-granfathers generation believed in most of the things because, you know "I heard it in the church". Our grand-grandfathers generation believed in most of the things because, you know "I read it on the newspaper". Our grandfathers generation believed in most of the things because, you know "I've heard it on the radio". Our parents generation believed in most of the things because, you know "I've seen it on TV". Our generation believed in most of the things because, you know "I've read/seen/watch it on social media".

I don't know about the next generations(s), the only thing in common for the above mentions was that they were made by a real-human.

Well, how is it done today? Twitter is instructive. You have mostly-automated accounts that like, retweet and reply to content from a specific perspective. Often it is an incendiary perspective, and usually it's a perspective underrepresented in regular news, because that's more useful for getting people to hit like and follow on the account's post.

Once the account has gained enough real followers, they can carefully start to push payload content, the content the operator really cares about.

This drives polarisation. There's this idea that sinister foreign adversaries are trying to "spread chaos", but I don't buy it. I think it's merely a by-product of building an audience. Russia (to pick an example) doesn't care about BLM, anti-BLM, US culture wars or the Assange case. Rather, it cares about exactly the things you'd expect it to care about: sanctions, conflicts it is involved in, allies etc.

(For that matter, ad-driven media does much the same. Gawker before its demise had really gone all-in on "outrage bait" stories.)

They're going to start with the most credulous, maybe, but they'll build confidence in the same way as everyone else who's an unknown at start.

> And there is no interpretation of that development as anything else but a net negative.

Sure there is: An inevitability.

I am hoping we will increasingly turn attention towards how to handle it (although I am relatively certain that's already going on at rapidly growing scale at fb, google and openai).

I could see updated legislation doing a lot of heavy lifting – strict rules against automated mass-disinformation – but more action is certainly going to be required.

The fact that something is "an inevitability" (which is definitely not the case here) has nothing to do with whether it's a net negative or not.

This is like saying "you should classify this as blue" and then a response like "no, it's heavy".

In actuality, it's more like saying "you can only classify this as blue", being awkwardly wrong with the premise, proceeding to not see reason and then get cute.

This has already happened, to a large extent.

Search for most any topic on google, e.g. recipes, and the first two pages will be chock-full of AI-generated copy.

Plot twist, people stop relying on news’ blogs’ meanings and create a system of decision making that is another level and actually works for them and not for the most sound speech talkers. You learn to swim by getting wet.

In the shadow end, typically. You can easily stand up and breathe.

Hot take: The majority of information consumed by people is compromised of low quality opinions, misrepresentations, and outright lies. It has always been this way, always will be this way, and every single person who clutches at their pearls over this fact also believes things would be improved by having the correct people control access to information.

This is the hydrogen bomb of propaganda.

Imagine assigning every single living human being a dedicated 24/7 con artist to follow them around and convince them of something. That's what will soon be possible if not already. It will be intimate con artistry at scale driven by big data, a massive DDOS attack on human cognition and our ability to conduct any form of honest discourse.

What hustlers, trolls, and completely amoral companies will do is bad enough. Now throw in state sponsored intelligence agencies, propaganda farms, militaries, special interest groups, and political parties.

Usually I'm anything but a luddite, but with this I can't help but think of many more evil uses than good ones. It doesn't help that the principal business models of the (consumer) Internet seem to center around surveillance, advertising, propaganda, and addictive forms of entertainment (like slot-machine-like mobile games) designed to suck money or time out of people.

Lesser but also very bad concerns include: the end of useful search engines due to a deluge of continuously learning adversarial SEO spam, the collapse of pretty much any open online forum due to same, and addictive "virtual friend" / "virtual relationship partner" hyper-sophisticated chatbots that hook vulnerable lonely people and then empty their bank accounts in various ways.

I really don't fear AI itself. I fear what human beings will do with AI.

> It will be intimate con artistry at scale driven by big data, a massive DDOS attack on human cognition and our ability to conduct any form of honest discourse.

This is an unfounded fear. For one thing, if the value in doing this is high then it's already cheap enough to be practical. The Chinese govt can pay millions of people to do this to dozens of people each. They basically do this already, for specific topics and issues. LLMs won't significantly move the needle here.

Second, are you proposing that attempts to stop Facebook from releasing models will somehow slow down or stop the Chinese, US, or Russian governments? What's the goal, to buy us 6 months? I would much rather the technology be out in the open for everyone to research and understand vs accessible only to state actors or huge tech companies.

The difference between this and a troll farm is like the difference between machine guns and lines of soldiers manually loading and firing muskets. Yes both can be used to gun down a lot of people, but machine guns are much faster and cheaper. Mechanized warfare is coming to propaganda and con artistry.

I'm not necessarily arguing for intervention to stop this release or something like that. The cat is out of the bag. There's no stopping it. This is going to happen, so get ready for it.

Oh, and throw in deepfakes. You'll have automatic con artistry at scale that can incorporate personalized fake audio and video on demand depicting any supporting detail it needs. It'll be like assigning each person a con artist who's also supported by a staff of content producers.

I guess, but on the flip side there are potentially transformative positive applications that we already know about and have yet to discover. Fundamentally some people are more optimistic and risk-loving when it comes to new technology. I believe the "good" will overwhelmingly outweigh the "bad" that you're pointing out. I think it mostly comes down to personality.

I can think of some positive applications. The thing that makes me cynical here is that all the evil applications seem like they're going to be far more profitable in terms of either money or power.

This would be a continuation of what's happened to the Internet in the last 10-15 years. The Internet is amazing and has tons of incredibly positive uses but all the money is in mass surveillance, addictive "engagement maximizing" stuff, and gambling and scams.

> Imagine assigning every single living human being a dedicated 24/7 con artist to follow them around and convince them of something.

This sounds like traditional Christian teaching of the role of demons.

We invented Satan and his minions to make people click ads.

History has shown that humans are terrible judges of the outcomes of our behaviour; your belief that we can confidently understand the risks of anything through experimentation might work in theory but hasn’t been borne out in practice.

Extremists exist at both ends of the spectrum and serve to balance each other out: without people positing the worst-case scenarios, the people positing the best-case scenarios would run full steam ahead without any consideration for what could happen.

Perhaps if the proponents of (various flavours of) AI were doing careful experimentation and iteratively working towards a better understanding, then maybe the loud voices against it would be less valuable, but as we’ve seen through the last 20 years, progress in technology is being made without a second thought for the consequences — and what Facebook are doing here is a bare minimum, so it’s reasonable for proponents to be somewhat cynical about the long term consequence.

It's much easier to sit on the sidelines and come up with reasons why you shouldn't do something new than it is to actually do it.

Some people have figured this out and built careers on it. This wouldn't be a problem, except that this opposition eventually becomes their professional identity - they derive prestige from being the person who is fighting against whatever. So even after researchers address their concerns, they have to ignore the progress or move the goalposts so they can keep on opposing it.

This doesn't seem like a bad thing to me. In the same way we have public defenders who professionally defend scoundrels, it seems good to have people who professionally critique new technologies.

I'm old enough to remember the naive optimism around the internet in the 2000s. "The long tail", "cognitive surplus", "one laptop per child", Creative Commons, the Arab "Spring", breathless Youtube videos about how social media is gonna revolutionize society for the better, etc. Hardly anyone forecasted clickbait, Trump tweets, revenge porn, crypto scams, or social media shaming. If we had a few professional critics who were incentivized to pour cold water on the whole deal, or at least scan the horizon for potential problems, maybe things would've turned out better.

Read Ted Kaczynski's (yes, that one) "Industrial Society and Its Future" with a neutral mind and you will start to understand why it's compelling.

My understanding is that when companies say “we aren’t releasing this / limited access / restrictions / for ethical reasons” they really mean “we aren’t releasing this because a) it’s expensive to run these models, b) it was even more expensive to create them and we might be able to profit, and c) maybe it’s bad for our ethics, which affects our funding and relations, and also, ethics.”

That's not the argument at all. Rather it's that the technology is progressing so fast and could become dangerous far faster than we can make it safe. Therefore it's worth seriously thinking about the risks that scenario poses. Stopping research or further work completely is A potential solution but given the monetary investments involved it's extremely unlikely it will be implemented.

There are lots of very serious people seriously looking at these issues and to dismiss them as simple luddites is frankly insulting.

But nobody is failing to take the risks seriously. The people who actually work on these models understand the risks far better than the outside ethicists. I work on ML in research. Reading their work is like listening to a drunk relative at Thanksgiving ranting about Bill Gates putting nanobots in vaccines. It's completely uninformed pseudoscience that comes from a place of strong political bias.

For example Timnit's "parrots" paper confused training with inference and GPUs with TPUs, making specific quantitative estimates that were off by orders of magnitude. If she had talked to a single person working on large language models, she would have recognized the error. But these people work in a bubble where facts don't matter and identify politics is everything.

There are enough people criticizing both current language models and the overall "quest" towards AGI that come from a non-political (unless you subscribe to an aristotelian everything-is-politics) perspective. I personally don't think any of the companies with significant AI research is actually doing anything meaningful in terms of safety. Also, from their public "utterings", it's quite clear to me that both Altman and Hassabis (not to mention Lecun) don't actually care about safety or consequences.

> I personally don't think any of the companies with significant AI research is actually doing anything meaningful in terms of safety

I assume this is just speculation on your part? Do you have any reason to make that claim? I personally know multiple people doing this full time at large tech companies.

I can give you some examples of serious safety oriented criticism of large language models to contrast with what plays out in the press and amongst "ethicists".

It's well understood that one can generate so-called "adversarial examples" for image classifiers. These adversarial examples can be chosen so that to a human they look like thing A, but the model classifies them as thing B with high probability. Methods of finding these adversaries are well understood. Methods of preventing them from being problematic are less developed but rapidly advancing.

For language models, the situation is much worse. I don't know of any effective way to prevent a large language model from being searched for adversarial inputs, trivially. That is, an attacker could find inputs from large curated input spaces that cause the model to output a specific, desired sequence. For example, an attacker with access to the model weights could probably find an innocuous looking input that causes the model to output "kill yourself".

Is this a risk that AI researchers are aware of? Yes, of course. But the difference between AI researchers and "ethicists" is that AI researchers understand the implications of the risk and will work on mitigations. "Ethicists" do not care about mitigating risk, and they don't care that the people who build the models already understand them and are comfortable with them.

>I can give you some examples of serious safety oriented criticism of large language models to contrast with what plays out in the press and amongst "ethicists".

To clarify I think the poster above was talking about the AI Alignment/Control Problem and not the specifics failure modes of particular models, LLM, CNNs etc. Very few people at OpenAI or Deepmind for example are seriously engaging with Alignment. Paul Cristiano at least acknowledges the problem but seems to think there will be available solutions in time to avert serious consequences which may or may not be the case. The folks at MIRI certainly don't seem optimistic.

>I personally know multiple people doing this full time at large tech companies.

The failure mode of internal "ethical" control at private enterprises is well-known and has already played out (at least) once when we tried to regulate medical experiments in the 2 decades after WW2. I personally consider the current AI safety positions to be just blatant whitewashing. The lemoine fiasco is a specifically hilarious case in point combining both a) a person that is utterly incompetent and biased to work at that position and b) total failure of leadership to adequately engage with an issue (or even admit it's possible in principle). At the current point, AI safety is roughly as useful as tobacco lobbying (exaggerated for effect).

Well I definitely wasn't talking about people like Timnit but rather researchers like Stuart Russell who actually are at the forefront of the field and discuss AI safety broadly.

> "ethicists"

> It's a political and emotional stance masquerading as a technical or scientific process.

I don't think you understand what ethics is.

I use to say that academic ethics is the study of what you can get away with.

If you want guidance on being good, you need a saint, not an ethicist.

Its a business play,

They are asking to be regulated because they have finished writing their models.

With regulation it will be harder for up and coming models to gain traction.

Its getting so much coverage because its paid press, I read about it in my newspaper BEFORE tech YouTube and here.

So...

I'm with you on being dispositionally pro-tech ad anti-luddite but....

>> lack of testable predictions coming from people like this

I think this is a disingenuous line of argument, more often than not. Popperian science is great, but it is not everything. The majority of our opinions, knowledge and conclusions are not based on testable hypotheses and falsifiable statements.

Take the sentence "One person should not have absolute power." It's not a falsifiable statement, or scientific in other ways. It's based on anecdote and folk wisdom, not science.

I agree with you. And, I think we need to argue back. But, don't argue meta. Meet the arguments head on.

It seems like there is a difference between "let's release it for researchers to investigate" and "let's release it for the general public to use and abuse, including all the script kiddies and malware authors and 4chan trolls and spammer sites out there."

Unfortunately, that difference can only be maintained through some kind of gatekeeping.

I like to try out the new algorithms, but I'm mostly just playing, and I don't see how they make it available to me without letting any random troll use it.

The movie Star Trek 2 has done tremendous damage to peoples' understanding of what a luddite was.

In modern terms, luddite wanted a tax on automation to support loss from tradesworkers who were too old to retrain.

It's similar to when bankers wanted a pension buyout at the introduction of the atm.

Luddites were pro technology, not anti. They just wanted wage distribution of the productivity gains.

They're basically early unions.

The attitude that I have trouble understanding is "A company spent millions of dollars researching and developing a new technology, they must make it available to me or else they are evil."

Spent millions on tech that could be a net negative for society. Keeping details secret makes people think they are evil because that's how coverups happen.

One day, these people will be right!

(And then we know one solution to the Fermi paradox.)

The remaining great apes remain alive primarily due to our pity for their plight- They were once the highest IQ species in the world, but no more

Probably, we will be in the same situation relatively soon: And there is little reason to expect the AI systems to have the same pity

Sorry I can't set up a double-blind, testable, peer-reviewed study to help convince you of this

Reply on news.ycombinator.com