Some signs of AI model collapse begin to reveal themselves
theregister.com
theregister.com
> That sounds good, but a "responsible AI user" is an oxymoron. For all the crap about how AI will encourage us to spend more time doing better work, the truth is AI users write fake papers including bullshit results.
I don't really think AI was ever meant to make us do "better" work, but more efficient work defined within a narrow set of parameters. Technology that scales encourages the optimization of single parameters, and the greater it scales, the more efficient that optimization is. For example, Google Search is a great example: huge scale, optimizing for SEO spam that is practically useless if you want to go into nuanced depth.
AI is even worse than that: it trades efficiency for accuracy, but efficiency at what? Production of things to be consumed, whether accurate, personally valuable, or not. After all, the internet has already tended towards this trend of production of the novel, at least for the average person. Yes, we can all point to subsystems that are nuanced and detailed such as some scientific paper repositories, but even science is being gamed and eaten up by the superficial.
We have instincts: to find new information, and that instinct itself is being gamed by AI that is perfect at finding the lowest-cost, lowest-quality information that is just so designed to stimulate our instinct of consumption.
In the short term, it does "help" us, but it also makes us less useful as "computational units", and thus more replaceable and slowly and subtly shifts our role from unique and creative souls to swappable computation units/consumers to direct AI. No one designed it that way, but it's the emergent direction of the system.
I understand what you're saying, but I hear all the time from AI boosters about how when AGI comes that it will let us do things like discover brand new theories, cure diseases, etc. (singularity, blah blah blah).
Even from the less hyperbolic types, I think a goal of real AI has always been "as smart or smarter than humans", despite what the current reality or limitations of LLMs are.
I get that. I do think there are three different levels we must distinguish between when considering this though:
1) What those who are selling AI want us to believe so we buy it
2) What those who are selling AI want
3) What AI actually will do based on its fundamental properties, and the emergent effects of mass human instinct operating on it
I was referring more to 3 and tangentially 2, not really to 1, which in my opinion is as relevant as a used car salesman telling you that that used Ford Fiesta in the corner is the most reliable thing you'll buy in the next ten years.
> I think a goal of real AI has always been "as smart or smarter than humans", despite what the current reality or limitations of LLMs are.
True, I think that's a goal of AI researchers, or at least AI computer scientists like the team who eventually build Deep Blue and similar folk. Though, the goal of the companies today are probably just "build it as good as it gets to get the money".
You can get really good results from search engines if you know how to use them and are willing to check the sources.
With AI as the search engine, you get what you get. This works well for a lot of searches, but not for everything.
Some engines, like Perplexity, Kagi and Gemini-enabled Google Search, will give you the sources they used in generating its response. This is fine, but for anything moderately complicated, this isn't enough, IMO.
For example: my wife was looking for a way to deter squirrels from eating out of our bird feeder. Gemini told her that we can use spicy bird feed to do this. It also provided a source. Great!
However, when you opened the sourced article, you'll see that Gemini's response completely failed to mention that not only is the spicy bird feed option a mild deterrant at best (squirrels will eat damn near anything if they're desperate enough), but it also failed to mention the two other MUCH BETTER options that were in the article! (Make the feeder free standing on a squirrel-proof pole, and some other option I forgot.)
I'm sure my wife could've told Gemini that the answer was incomplete to get a better response out of it. However, that perfectly underscores the problem I have with the tech in its current state: you're getting some inaccurate representation of the truth while also stealing market share from the authors and companies that shared their knowledge online to begin with.
I also don't think AI search is SEO proof either. People will figure out how to bend Gemini to its will. Given that most people will use natural language queries, the Wendy's [] singularity is almost inevitable.
[] Wendy's, a fast-food chain in the US, has an excellent social media team. I wouldn't be surprised if they find a way of getting every Gemini/Bing OpenAI search to work Wendy's in somehow!
This isn’t a huge problem with search as it existed 5 years ago. You’ll see the URL and say, “That doesn’t sound like a reliable source. I’m going to scroll down a little further.”
But with AI search, the authorship is less obvious. It looks like the AI is saying it, rather than the AI just mashing together a bunch of search results with some garbage in the mix. It allows creeps, kooks, and nutballs to launder their ideas by getting AI to repeat their garbage.
It was a concerning interaction, and I worry how many impressionable people are being fed weird cult-spam while thinking that it’s legitimate because they don’t realize where it’s coming from.
That's a good point. Stagnant performance would be a different thing -- more indicative of the technology itself.
That said there is a clear vulnerability in LLM's not being able to judge whether information is true or not especially when pulling from the web.
Please omit swipes like this from comments on HN.
You can re-apply the process later to move users to higher tiers.