Generative AI could make search harder to trust
wired.com
wired.com
I had found some silver ingots. The top search result for "bg3 silver ingot" is a content farm article that very confidently claims you can use them at a workbench in Act 3 to upgrade your weapons.
Except this is a complete fabrication: silver ingots exist only to sell, and there is no workbench. There is no mechanic (short of mods) that allows you to change a weapon's stats.
I'm pretty sure an LLM "helped" write the article because it's a lot of trouble to go through just to be straight up wrong - if you're a low effort content farm, why in the world would you go through the trouble if fabricating an entire game mechanic instead of taking the low effort "They exist only to be sold" road?
This experience has caused me to start checking the date of search results: if it's 2022 and before, at least it's written by a human. If it's 2023 and on, I dust off my 90's "everything on the World Wide Web is wrong" glasses.
They currently manufacture these karma rich accounts by reposting popular posts and comments. LLMs will soon be (or already are) another way to karma farm.
It's the typical overvalued VC-backed company dilemma that needs investor returns. Quora, Medium, and so on.
Similarly, if it costs basically nothing to work your way into communities to astroturf with bots, it'll happen. You don't have to post about great sites to get free Viagra right away, you can build reputation and subtly astroturf. And you can use additional bots to build/portray consensus.
Reddit is already a problem because of actual humans doing the latter. It'll just get worse when it's automated further.
because misinformation written by humans didn’t exist before LLMs?
There's "one loony had a blog" levels of wrong, and then there's "industrial scale bullshit" levels of wrong and we are not prepared for the latter.
In the 00s and 10s, the quality of discoverable content improved: reddit and stackechange had experts (at a higher rate than the rest of the net at least). It was the era where search was good, Google wasn't evil (good results that separated the ads are entirely why they won against AskJeeves and Yahoo), and SEO was still gestating in Adam Smith's wastebasket.
Now Google and Bing are polluted with SEO-optimized content farms designed to waste your time and show you as many ads as possible. They hunger for new content (for the SEO gods demand regular posts), and the cheapest way to do this is an underpaid "author" spewing out a GPT-created firehose of content.
SEO has ruined search, and content farms have made what few usable results there are even less trustworthy.
So yes, the Internet has fundamentally changed in the last 9 months.
With Reddit you might get inane arguments and bandwagoning about what the best game strategy is, but you're exceedingly unlikely to read about a game mechanic that was straight-up hallucinated by a LLM.
Before LLMs, a $3000 camera had fake reviews on Amazon, and you got fake news about politicians. But you can safely assume "bg3 silver ingot" information is likely real, since hiring someone to make up silver ignot will never make the money back.
No any more.
Kagi's ability to manually downrank/remove those kinds of results from your searches (and their return to flat rate pricing) finally tipped the scales for me for subscribing/switching search.
> In Baldur's Gate 3, silver ingots are a common miscellaneous item that can be found in various locations such as chests, shops, and dropped by enemies.[1] Each silver ingot can be exchanged for 50 gold at merchants or traders.[2] While silver ingots do not have any crafting or upgrade uses currently in early access, they provide a reliable source of income early in the game before other money-making options become available.[3]
Their argument is that since it's centralized, things like that are possible (while with llama2 you can't), they do "patch" things all the time. But since blobspam are contributing to paying back the billions microsoft expects they're not going to.
Unfortunately, any question that lots of people legitimately ask will also be a prompt for blogspam.
If it's a perceptual hash, that's easy to _exploit_: just ask the AI to repeat something back to you, and it typically does so with few errors. Now you can mark anything you like as "AI-generated". (Compare to Schneier's take on SmartWater at https://www.schneier.com/blog/archives/2008/03/the_security_...).
Also, it would require OpenAI to store the history of everything that it's API has produced. That would be in contrast with their privacy policy and privacy protections.
Back then I was playing Hyperdimension Neptunia and almost tried “applying AI” in the old sense to the problem of “What do I have to do to craft item X?”. Those games all had good FAQs and extracting a knowledge graph and feeding it into some engine that could resolve dependencies wouldn’t have been too hard.
Today I am playing Atelier Sophie which has the same mechanic but is very cozy and doesn’t pose complex dependency problems and the FAQs for this game are atrocious, consisting of a walkthrough that way too prescriptive. If you ask some question like “Where do I get a Night Crystal?” on Google this is likely to turn up a question/answer pair on a forum which isn’t quite as good as having the structured FAQ.
YouTube walkthroughs really seemed to kill text walkthroughs, sometimes these are better (like when there is a jump in a dungeon that doesn’t look like you could make it but you can) but sometime they are much worse (there are 75 hour long videos in a walkthrough, you have to find that it is in video #33 and that you have to seek to 20:55.)
Maybe the proliferation of trash sites will motivate the creation of high quality FAQs but you’d better believe that the creator of these FAQs will be horribly afraid of being ripped off.
That being said how many people write blogs with grammerly or chatgpt these days. The temptation to use these technologies all the time is too strong for even self preservation of your own (writers) voice.
My sense is that you use this technology you might be happy with the results at first but on later review you just notice something off in some sentences and maybe it just doesn’t flow right. I’m not convinced that it will replace writers jobs yet. Especially when you want to create something authentic and unique.
I use ChatGPT all the time to suggest how I could make sure something isn't passive aggressive. It'll point out parts that aggression and suggested changes. It can be for a short slack message, or a many paragraph message.
I don't know about that. I have played with ChatGPT/Copilot/etc enough to know what they're capable of doing. But the thing is, I enjoy programming. I enjoy breaking down a problem and solving it with code. I enjoy crafting elegant code. So I don't use AI even though I'm fully aware it could save me hours on projects. Why? Because I enjoy those hours very much.
Why am I telling you all this? Because I suspect many writers are the same and personal blogs are their canvas. They enjoy communicating. They enjoy crafting articles. They might have AI proof-read them, but they won't let them write everything. So, to me, there is hope that personal blogs will maintain their human element, as opposed to news websites or tabloids or learning platforms.
Enjoy this luxury while it lasts. Based on what I have seen in performance review committees for software developers, your peers who drive results faster than you do because they use AI will be rewarded more and will be more likely to survive rounds of layoffs when they inevitably happen.
If the job changes that drastically I'll just have to quit and find something else.
There is a small sub-plot about how he had to give a fake persona credibility on the untrusted network in order to be able to leverage a creating a fake account on the trusted network.
Explained: <https://www.explainxkcd.com/wiki/index.php/635:_Locke_and_De...>
Reality has largely demonstrated that far more thoughtless propaganda of the Big Lie, Firehose of Bullshit (or Falsehood), associated with Russia, floods of irrelevance which tend to bury more significant stories, favoured by China, and outrage / hot-button topics, which are common in US-centric media, though a timeless technique.
Memes and simple messages attract attention and spread. Complex narratives and analyses ... not so much.
But yes, voices that deserve no attention whatsover have dominated the media landscape of the past decade or so. Not that this is entirely novel.
But yeah, maybe the idea that you can even 1% trust random content on the Internet without having a source doesn't really make sense if you think about it IMHO. Either you do this web of trust, coming from a well know real world source, or be Wikipedia-like with linked reliable sources for the viewer to check.
By the way, wasn't this how Google ranked pages back in the day? Ranking pages that get linked to higher? And even before that there were P2P web rings.
At a minimum, you'd have to validate them by confirming existence in the Wayback Machine.
Otherwise agreed that those are indeed high-signal documents. Increasing reliance on integrated educational software means that even such things as online syllabi are increasingly rare.
.edu domains can be had for any otherwise eligible "U.S.-based postsecondary institutions" per Educause: <https://net.educause.edu/eligibility.htm>
Pages at extant domains might variously be available to undergraduate or graduate students, faculty, staff, and adjuncts. Those might either directly host emulative material or be convinced or compromised into hosting content.
If there's one thing that the Internet's history to date has proved, its that perverse incentives lead to perverse consequences.
- Enroll or be hired at an eligible institution. There are literally thousands of these.
- Bribe or compromise someone enrolled or hired at an eligible institution.
- Create a de novo eligible institution. For-profit colleges are not uncommon.
Someone motivated by profit or advantage would likely find virtually any of these options quite straightforward.
I'm ... somewhat pained that this needs to be spelled out.
You don’t generally get the kind of personal website being discussed to my understanding.
> Bribe or compromise someone enrolled or hired at an eligible institution
Finding a professor willing to stake their job and reputation for such a blatantly immoral scam seems hard.
> Create a de novo eligible institution
Is it actually easy for a regular person to create their own college?
I find the snarky finish to your comment obnoxious.
- start their own country and call it Edunistan
- bribe ICANN to take over the .edu TLD
- open a university in the new country
- spend 15 years earning a PhD at that university
- reserve ~/name and start posting LLM generated content
Browsing today is like: “You ask for a spaghetti recipe and the page tell you the whole history of civilization.”
Also, note to self to collect my favourite recipes in markdown files from now on.
Allegro (big polish auction/e-commerce site) in their mobile app will unconditionally rewrite the search terms instead of showing you no results
As time goes on, even the amount of text I am putting out get trimmed down. Make the words count, don't count the words.
It seems to me it relies mostly on discounting just how much we've already had to deal with this same problem in humans over the millenia.
The problem of proliferation of bad information might be getting worse, but this isn't native to generative AI. The entire informational ecosystem has to deal with this. GPTs compound the issue, but as far as I can tell, no where near what social media has forced us to deal with.
A human's lie is different than an AI's hallucination, since it's still based on (distorting) the truth, whereas the hallucination is based on an invented reality (yes I know it's applied statistics and there's no true model of the world in there, but it can report as if there is)
LLMs are no different in this respect.
LLMs on the other hand are amazing and prolific liars and can produce a lot of bullshit for a price that’s effectively free - in fact it’s cheaper to create a LLM that’s inaccurate than one that’s … less inaccurate. The truth to lie ratio on the internet is about to take a huge hit. LLMs really are a Pandora’s Box.
That soft data could have never been trusted, rhe information that can be verified (calculations etc.) seems safe from LLM
with AI this limit is all but removed
all the human generated bullshit ever created will soon be dwarfed by what AI can vomit out in an hour
LLMs changes all that. They can produce content on a massive scale that can drown out everything else. It doesn’t help the fact that more inaccurate LLMs are cheaper and easier to create and run - think about all the ChatGPT3 level LLMs vs ChatGPT4 level LLMs; guess which ones SEOs will gravitate towards.
> Because everyone has at least tried to lie - not necessarily for malicious reasons, e.g. white lies - a few times in their lives and know it’s not easy coming up with plausible sounding lies.
How do you know that?
Common knowledge.
Regardless, the amount of false information on the internet used to be limited by the amount of people producing it. Some of it is deliberately produced misinformation. Some of it are just mistakes due to carelessness, others due to willful negligence - example of the latter would be content farms who make their money off flooding search engine result pages and pushing ads in your face when you visit their website; who couldn’t care less what if you got what you came for.
Despite all that the internet was still useful. There are enough people posting accurate information such that the signal to noise ratio is good enough.
LLMs will change all that. They have the potential to flood the internet with hallucinated falsehood. And because bad LLMs are cheaper and easy to create and operate, those will be the majority of LLMs used by aforementioned content farms.
They won't make up a lie when telling the truth is far easier and when they do have to lie, it's a slight bit more effort.
But that's a moot point with LLMs ...
If/When they use LLMs, they will pick the cheapest LLM possible which will produce the most garbage - they might not even bother keeping it up to date; why bother when hallucinated BS sound just as convincing.
It has already started to happen.
Humans always assemble information according to a standard of truth. It is a big part of how humans learn. No method is perfect but the human method results in fewer routine hallucinations.
Embodiment and multi modal AI will likely provide such filters or limits to AI in which it can derive truth.
I'd love you to produce data to back this up.
My guess is that you are wrong, on the basis of how often I discover that I'm full of shit and how often I discover other people are full of shit.
Humans are built for being wrong just as much as being right. We wouldn't have such complicated institutions and social structures built around controlling for those symmetric capacities if it weren't the case.
Even then, we find ourselves surrounded and overcome by falsehoods of our own design.
In doing so the species would improve critical thinking skills which can be applied to all information regardless of source. Which, I agree, was often BS to begin with. But in theory would be more difficult to skirt on by without notice if humanity upgraded their critical thinking.
What I'm responding to is the strong tendency to discount our very long history of dealing with factually incorrect information and the ascertainment of truth from sources both dubious and trustworthy.
Entire institutions are set up in order to handle these very real problems, the set of which currently dwarfs the problem of hallucinations in GPT.
From a social perspective, non-GPT falsehoods are even more insidious, because we are inclined to trust and believe those whom we like and are like us.
Again, people are in the habit of discounting just how much we are wrong in our everyday lives. The hallucinations therefore appear more singular than they actually are.
My inner conspiracy theorist can't help wonder if the continued reduction in search usefulness isn't part of an ongoing deliberate disempowerment of everyday people - but my rational side says it's merely an unfortunate emergent behaviour of the systems we've built.
"Oh, you want a guide to writing your own loss function for Tensorflow? Here's an FAQ that could have existed on comp.lang.python3.tensorflow"
Does this mean we'd end up with a finite set of verified human only data?
Would people start going through all kinds of offline archives via AI-gapped means, trying to uncover and document new sources of human input?
Assuming that semi-convincing misinformation spreads everywhere, people will finally have to find the original source of a certain statement, verify their "knowledge supply chain", and maybe use logic to evaluate every single statement made.
But who's gonna pay for it?
https://boingboing.net/2023/10/05/ai-search-chatbots-output-...
Web directories , 'Who's Who in Engineering' type lists, etc.
It's a step back from universal search engines being able to find stuff, but it's a step forward with regards to curation and quality of results; so i'm not sure if it's entirely a downgrade.
The early 90s 'website phonebook' type encyclopedias were interesting[0], but I always had to remind my mom "No, this isn't the entire internet, it's just a bunch of places that people like; the secret ones are 'unlisted'."
Note: I never say this is better than a search engine, it's just an interesting end-result after search engines got polluted and modified til the point of uselessness that we're at now with Google.
[0]: https://www.amazon.com/Internet-Directory-Guide-Usenet-Bitne...
- Wikipedia
- Online documentation for whatever language/framework/tool I'm using
- Stack Overflow / Stack Exchange for most technical questions
- Reddit if SO/SE doesn't work, and for opinionated questions (e.g. r/BuyItForLife)
- Hacker news for software recommendations and technical opinionated questions
- Arxiv or the ACM library if it's a research paper (99% of the time, whenever I google something niche the only relevant results are papers)
- Other sites like caniuse.com, university sites for health and nutritional info, old-style forums for specific software
For these searches I'm just using Google to bring me to the specific site I want, because it's faster than using the site's own search functionality. Then there are the times I literally just type in the website instead of the URL bar (e.g. "instacart"), or when I use Google maps, images, or reviews.
I'm always wary when Google returns an unfamiliar site because I'm skeptical of the results. ~70% of the time it's some blogspam which is at best accurate but overly wordy, and at worst inaccurate; sometimes it's a blog from some random individual who for whatever reason went into a deep dive trying to understand what I'm searching for, that actually turns out to be useful; the rest, idk.
After finding the contributed article (on a well-known news site, not Wired though), it looks like a tech founder might've been using ChatGPT to write an article about the uses for WASM. The arguments were generally sound, but I don't think that anyone did the work to manually check any of the facts they presented in it.
It already did, even in the "purely human" era. I think LLM text will gradually become more trustworthy than a random website by consistency filtering the training set.
Now I hear of people discovering they're in prison, married to random people they've never met, or are actually already dead.
What is this going to do to recon on individuals (for example by employers, border agents or potential romantic partners) when there's a good chance the reputation raffle will report you as a serial rapist, kiddy-fiddler or Tory politician?
Increase noise to drown signal.
In the near future, the web could become opaque with LLM schlock, but at least it may grant people a right to be forgotten.
[1]https://www.businessinsider.com/lindsey-stone--so-youve-been...
I don't think it worked.
My real name is very, very common -- so this has been my reality for my entire life.
These days, I have grown to appreciate it. It's like an invisibility superpower.
I feel terrible for the first dozen people this happens to. That said, I look forward to this being the average case for most people. Bury the real malfeasance in AI-generated noise. Let employers and background checkers get dunked on.
All three in my case, and that just for sites which predate AI.
This is too much of a temptation for the SEO scum to resist.
Ultimately, I would like to see more about the other side. If generative AI can make blog spam, then I think it can recognise blog spam. How far are we from implementing a reliable filter of useless spam sites from search results? I don't expect Google has a monetary incentive for this, but maybe someone else does. But from my story above maybe "search" is a thing of the past and for functional queries, we will just talk to the gatekeeper. Reading the actual words written will only be for leisure.
The worse organic results are, the more people will click on paid links. This is WHY everyone on HN is complaining about search results, because google doesn't really have an incentive to give you really good results. They only need to be good enough to keep 95% of the population still using google, but mostly expecting the good results to be ads.
Google ads are the equivalent of verification on FB and X. They just call it something different. The verified, high quality results will be paid.
Its not simply garbage in garbage out. There is no logic to verify and analyze the data. You are simply told what is popular in the data.
It's a core problem with generative AI and it can't be solved with better data.
I'm starting to wish articles had inline citations as a standard.
Or would footnotes / sidenotes be acceptable?
Its xkcd's Citogenesis automated and at internet scale https://xkcd.com/978/
>Web search is such a routine part of daily life that it’s easy to forget how marvelous it is. Type into a little text box and a complex array of technologies—vast data centers, ravenous web crawlers, and stacks of algorithms that poke and parse a query—spring into action to serve you a simple set of relevant results.
Web search has, for me, become a nasty twisted hall of mirrors well before generative AI. I almose never get fed relevant results, I alsmost always have to go back and quote all my search terms because the search engine decided it didn't really need to use all of them (usually just one.) The only difference is the poison was human generated. generative AI will simply erase the 5% of results that might give me an answer quickly.
I just had to google a bunch of races that I wanted to run. The top result was always the event's own web site.
When I google some news, relevant news articles always come up.
The last search I did was for how to display a ket vector in LaTeX. The top result was the StackExchange article with the right answer.
From what I see, certain domains seem to be targeted for exploitation. Programming questions seem to be high up on the list. I wonder if that skews HN readers' perceptions.
Google search to retrieve anything opinion related has been horrible and infested with blogspam for years (hence people searching Reddit to get that kind of info).
Google is a giant adware tool that’s been taken over by adware SEO sites. The example given - find the product marketing pages for some races - falls directly in its sweet spot. If you venture outside it’ll do its best to get you back into the product marketing sweet spot, and the SEO companies of the world take care of the rest.
Search is a lost cause.
I'm more likely to start at a place that aggregates reviews and try to hallucinate which ones were written by people who know what they're talking about. That usually seems to work.
I imagine that somewhere out there is a person who bought the product and reviewed it on their blog or made some enthusiastic social media post about it, and that's what you'd want to locate were it not for the spam. But I don't expect any search engine to be able to find it for me.
Nothing will change as long as search is optimized for revenue over user value.
Interestingly, we are in spot right now where I feel that for certain types of queries LLMs can outperform search engines. But from what is shown in the article, it seems like that state might only be temporary, and that in the same way that shitty content farms mastered SEO and polluted search results, we might see the same happening with LLMs that have access to the Internet.
Adding "reddit" to queries can be pretty useful. You're prone to get terrible, inaccurate information since it's just random people on an internet forum, but at least it's (usually) actual humans and not blogs trying to SEO-game. (Though one big caveat is searching for products/services. Lots of threads full of bot accounts writing "[link] has been the best [thing], in my experience". They're usually easy to spot, but sometimes they do seem pretty natural until you check the post history.)
Less and less so. Reddit has always had a bot problem, but it seems to be getting exponentially worse lately. Not just article reposters, but comment reposters, bots that reverse images and videos just to repost, seems like it's at least 75% bot content now.
However, there always is a sufficiently reliable/checkable fact: "a specific media wrote...". Whatever they actually wrote is not a fact but they wrote it - this is. This said you can then ask youself why they most likely did, keeping in mind other observations of yours.
If LLM usage in media becomes widespread, I'd pay for a service that identifies and hides the LLM shit for me.
There still exists a problem that users need to run and manage their own indexes, at times.
So search engines in their traditional sense will be obsolete anyway.
1) GPT-4 and other such LLMs will generate textbooks and manuals for every conceivable topic.
2) These textbooks will be 'dehallucinated' and curated by known experts on particular topics, who have reputations to maintain. The experts' names will be advertised by the LLM provider.
3) People will search for stuff by chatting with the LLMs, which will in turn provide citations for the chat output from the curated textbooks.
But because of its current lack of optimization for accuracy, we shouldn’t consider it disruptive because it’s not yet proven technology?
You can call it dangerous but you can’t call it useless. It’s also only going towards improvement from here, including drastic reductions in hallucinations.
You have to remember too that AI models are generally attempting to interpret the intent behind the prompt, so many of these crazy articles are happening because people aren’t yet good at writing clear instructions for AI and AI isn’t yet mature enough to disambiguate poor instructions in its output and is trying to deliver on unclear instructional intents.
Why?
> Why?
A pseudo-religious belief in progress, especially the technological kind.
It's bullshit. If it were true, 2022 pre-LLM Google would have been better than 2010-era Google, but it most definitely wasn't. Consumer printer technology, for the most part, has been getting worse for decades at this point.
Misinformation and disinformation is already a problem on the web and thrusting unproven technology like generative AI that has a tendency towards misinformation is opening a Pandora's box. But as long as Microsoft and Google and Meta can make their money...
(shameless plug) At Metaphor (https://platform.metaphor.systems/), we’re building a search engine that avoids SEO content by relying on human curation + neural embeddings for our index + retrieval algorithm. Our mission is to ensure that the information we receive is as high quality and truthful as possible as AI adoption marches onwards. You (or your LLM) can feel free to give it a try :)
There are good critical viewpoints but most of the articles they are putting out at this point read like bitter diatribes. Which is a shame because they used to be an excellent publication.
The academic internet of the 90s is so far gone and while we're seeing a lot of magic lately, it's magic available to literally everybody for any and every purpose.
We're rapidly seeing how boring and disappointing that is :(
The problem isn’t that Wired is critical, it’s that they’ve gone weirdly reactionary and their writing has gone so mass market dumbed down that Some Random Guy’s Blog is likely to have a better written and researched viewpoint.
Essentially, the comment made the point that tech is advancing so fast these days that most people are unable to keep up with the pace of these radical changes. And the natural reaction to that for many is to reject these "advancements" or at least look upon them with cynicism and skepticism.