Almost all of the "product X vs Y" results are AI ramblings now. This growth of the dead Internet is making me want to sign up for Kagi. We're going to need a certification for human generated content at some point.
As mentioned, product comparisons are a big one but another worrying area is anything medical related.
I was trying to find research about a medicine I'm taking this week and the already SEO infested results of 5 years ago have become immeasurably worse, with 100s of pages of GPT generated spam trying to attract your click.
I ended up ditching search alltogether and ended up finding a semi-relevant paper on the nih.gov and going through the citations manually to trying and find information.
For content that isn't time sensitive the best trick that I have found is to exclude the last 10-15 years from search results. I've setup a Firefox keyword searches[1] for this, and find myself using them for the majority of my searches, and only use normal search for subjects where the information must be from the last few years. It does penalize "evergreen" pages where sites continuously make minor changes to pages to bump their SEO, which sucks for some old articles at contemporary sites, but for the most part gives much better results.
[1] For example: https://www.google.com/search?q=%s&source=lnt&tbs=cdr%3A1%2C...
OMG. I'm so happy how much AI is improving our lives right now. It really is the future, and that future is bright.
Thanks guys!
Have you reported any of those issues to Kagi (support/discord/user forum)? We are pretty good at dealing with search quality issues.
I’m not affiliated with Kagi in any way and my results are not full of LLM generated content either.
Though there is an LLM generated baby peacock pretty high in the image search, and when I go to the website it literally says that it is an example of an AI generated image and not a real baby peacock.
I've been doing this for years now. The normienet as I call it is nigh worthless, and I don't even bother trying to find information on it.
I wouldn't mind a modern take on geocities sorta system. Where: (1) You can make a webpage that could be about whatever the bleep you wanted it to be about. (2) Only allowed a reduced subset of web technologies. (3) That was free from any advertising or commerce/sales. (4) Was only available to individuals or businesses no larger than closely-held corporation. (5) had clear limitation on AI uses. (6) Had a complete index, categorized and tagged, of all the sites available.
But if I am being honest, that is just the nostalgia for the old internet talking.
This is not going to work - as time progresses, there will be less and less nostalgic people who are willing to put up with that complexity. And "non-commercial" part will ensure that there _never_ be an option to say: "I am tired of fixing my homeserver once again, I am going to put up my site to (github|sourceforge|$1 hosting) and forget about it".
Compare to early web. First thing that came to my mind was Bowden's Hobby Circuits site [0]. It's designed for advanced beginners - simple projects, nice explanations. And there are no hoops to jump through - I've personally sent the links to it to many people via forums, private emails, and so on. It apparently went down in 2023, but while it was still up, I remember regularly finding it from google searches and via links from other pages.
[0] https://web.archive.org/web/20220429084959/http://www.bowden...
I'm ready to pay for a walled garden where the incentives are aligned towards me, instead of against me. I know that puts me in a minority, but I'm tired of the advertising 'net.
Bring it back. Charge me 10 or 20 a month. Give me the walled off chatrooms, forums, IM, articles, keywords, search, etc. Revamp it, make it modern. And make a mobile app.
Everyone wanted a free and open Internet, until AI and the bots ruined it all.
Well... advertising as a business model ruined it all. They get paid for getting page views, so the business model optimizes for maximum page views at minimum cost of creation. This is the end result of what Google and Facebook have spent the last 20 years building.
But I'm sure the engineers who built all this have very nice yachts, so it's all fine.
It's largely already happening in places like Discord.
I think the first company that can capture what Discord has but is not wrapped in that "gamer aesthetic" ui/ux is going to do really well.
[1] https://www.tomshardware.com/news/discord-throttles-nvidia-g...
[2] https://www.thestreet.com/technology/discord-is-making-a-maj...
And eventually someone is going to come up with a search engine for Discord, and the cycle will start all over.
It's great to read these words. People are starting to get it. The Internet is not for you, it's against you.
Patreon, forums, Discord, Facebook, Instagram and so on are all centralized.
With the Internet of the 90s, discussion happened with decentralized Usenet (owned by nobody) with more-focused discussion happening on mailing lists (literally owned by whoever was smart enough to get Majordomo compiled and running).
Email was handled by individual ISPs or other orgs instead of funneled through a handful of blessed providers like Gmail.
Real-time chat was distributed on networks of IRC servers that were individually operated by people, not corporations.
Quickly publishing a thing on the web meant putting some files in ~/public_html, not selecting a WordPress host or using imgur.
Ports 80 and 25 were not blocked by default.
Multiplayer games were self-hosted without a central corpo authority.
One could construct an argument that supports either way being better than the other, but the the Internet of today is not the same thing as it was a quarter of a century ago.
(Anyone can make a "discord server," but all that means is that they've placed some data in some corpo database that they can never actually own or control.)
The technology doesn't matter, it's changing all the time. The difference is the size of the community, and the barriers to entry. If these communities were easy to discover and join and hard to be booted out from, they wouldn't be protected from The Slop. It used to be just getting on the Internet was the barrier. Now we need new barriers: paywalls and word-of-mouth and moderation.
But AOL IM and MSN were centralized walled gardens that came rather late to the game, and neither PhotoBucket nor phpBB existed at all in the 20th century that is the context here.
Losing "dedicated servers" was a huge loss in my opinion. It was fun to play on the same handful of servers and get to know the same group of people. Dedicated servers were also free from profit-driven "matchmaking" schemes since you ended up playing with whoever was on at the time.
I also miss the chaotic multiplayer of the era, where the priority was having fun and not improving your rank on the leaderboard.
Edit: Dedicated servers also each had their own moderation. So if you wanted to play on a server that banned anyone nasty you could. But you could play with a bunch of folks dropping "gamer words" as well.
Then there were custom sprays - something no company would allow in their game today. It's sad how constrained and censored online gaming experiences are today. In many games even dropping the "f-bomb" can get you banned and typing assassin in the chat window yields "***in".
If you want to find it again, change your search engine. Use wiby.me, or search.marginalia.nu. Subscribe to RSS feeds of sites you find interesting, and go from there. Hop on gemini. Subscribe to some activitypub accounts (you don't even need an account for that). Communal IRC servers still exist, forums still exist, independent emails still exist.
Stop falling for the overall doomism HN is so quick to fall for, start doing and living what you want
1. could afford a computer back then and saw the utility of owning one. 2. had access to the internet, so either in college, a 'tech' company, or ties to some local collective that provided access.
When people say 'the old internet' they are referring to a very self selective/elite group.
And that's what made it fun I guess
I'm a little sad for anyone who didn't get to experience of pre-Internet era.
Internet is lead of our time.
I would. But I've asked a lot of people who say "no, I don't want to pay when I can read it for free. I don't mind the ads that much."
Sadly, they won't know what they were missing. It'll be the new normal
Some asshole tech apologist is probably getting ready to post that section from Plato where Socrates complains about writing any minute now.
Of course, that asshole is oblivious to the fact that most if not all of us probably just don't understand what Socrates was missing, so he's just showing his ignorance and stupidity.
> I'm ready to pay for a walled garden where the incentives are aligned towards me, instead of against me. I know that puts me in a minority, but I'm tired of the advertising 'net.
The problem is that, even if you try to do that, the incentives are probably still aligned against you, just maybe less blatantly.
Just look at how many formerly ad-free paid services are adding ads, and how hardware users literally own acts against their interests by pushing ads in their faces (e.g. smart TVs).
The guy who runs the walled garden will always be tempted to get some extra cash by adding ad revenue to your subscription feed, or cut costs by replacing human curated stuff with AI slop (maybe cleaned up a bit).
I did, and...well, let's be careful how we look back at it.
Punch the monkey? Ad supported 'free' internet that literally put an adbar at the top of your browser at all times? Dreadfully slow loads of someone's animated construction sign GIF? Waiting for dial up to connect after 20 tries? Tracking super pixels? Java web applets? Flash? Watching your favorite ISP implode or get bought up? To say nothing of the pre-Google search results (I miss the categories though).
I have plenty of good memories from those days, but it still had plenty of problems. And it wasn't exactly a bastion of research material either unless you really went digging or paid for access.
But my point is, the latter books have this has amazing post-internet, just a ravaged chaotic Wildlands filled with rabid programs & wild viruses. Packets staggering half intact across the virtualscape, hit by digital storms. Our internet isn't quite so amazing, but I see the relationship more subtly with where we have gone, with so so so many generated sites happy to regurgitate information poorly at you or to sell you a slant quietly. Bereft of real sites, real traffic. Watts is a master writer. Maelstrom.
First book Starfish is free. https://www.rifters.com/real/STARFISH.htm
Another book on my reading list!
User facing ability to whitelist and blacklist websites in search results, ability to set weights for websites you want to see higher in search results.
Spamlists for search results, so even if you don't have knowledge/experience to do it yourself, you can still protect them from spam.
It's recreation of e-mail situation, not because it's good, but because www is getting even worse than e-mail.
It is quite surreal to witness. It is certainly fueled by the commercialization of internet due to ads and centralization to user hostile platforms.
The old internet seems to be doing much better. But it lost most of its users in the last 15 years...
What do you mean by this? How do you find the old internet?
Many of e.g. the old niche forums still exist. Like, FOSS sites. GNU project sites seemed not to have aged a day in 20 years, i.e. still party like its 2004.
Also, I think non English sites are better off since Reddit mainly ate English communities and sites.
Facebook is probably what killed most of the living internet. Small community sites. Like the local Kennel club or Boat marina.
A good example of the old internet would be Matthew's Volvo site:
https://www.matthewsvolvosite.com/forums/search.php?search_i...
https://www.reddit.com/robots.txt
Reddit serves a different robots.txt to Googlebot, you can see a snippet of it in Googles summary. If Kagi is getting recent Reddit results then they must be either ignoring robots.txt, using Google as a backend, or also paying Reddit for access like Google.
Bing and DuckDuckGo are certainly still locked out, I just tried searching for "reddit hurricane milton" on both and none of the results are actually from Reddit.
https://help.kagi.com/kagi/search-details/search-sources.htm...
The whole point of the API access change was to charge AI model-makers. I'd be ironic if the API change made destroyed their product and made their data unsellable.
Recently, they came around looking to recruit me. I told them fire Spez or fuck off. (17 year Reddit user here)
I think if the business model had been thought in the sense of the communities and involving mods and users in, it would have been genius, a lot of smaller companies would kill to have people genuinely recommending their products/tools that are hidden behind the biggest wall of them all...
Alas they completely ignored this as a viable avenue and went for the quick buck.
But this won't last and at some point people are going to move on from mass scraping, either because they already got what they want or because garbage goes in garbage comes out or because most of the content will be bot generated and require too much filtering to be useful.
Of course this is the opinion and rambling of a moderately educated individual and I might be totally wrong.
Change can often be for the best, pretty sure it wasn't in this case...
Aaron is probably turning once again in his grave...
I have no idea how intensive these are, as I most of these over bar drinks when I was traveling. Could've just been someone making stuff up as well. But my marketing friends have confirmed that they use Reddit very heavily, as it's a great sales funnel if you play it right.
[0] Gradually for several years already.
I guess using the correct terminology matters.
If people were actually searching up "peachick" that'd probably be SEO spammed to hell, too.
"Google sucks!"(50 upvotes)
"That's why I use Kagi!"(45 upvotes)
"Actually Kagi has the exact same problem and you have to pay for it."(2 upvotes)
People keep saying this and I keep warning them to be careful what they wish for. The most likely outcome is that "certification of human generated content" arrives in the form of remote attestation where you can't get on the internet unless you're on a device with a cryptographically sealed boot chain that prevents any untrusted code from running and also has your camera on to make sure you're a human. It won't be required by law, but no real websites will let you sign in without it, and any sites that don't use it will be overrun with junk.
I hate this future but it's looking increasingly inevitable.
https://x.com/samhenrigold/status/1843040235325964549
...or a sensical comparison where it just completely misses the point.
Edit: Similar to the second one I just did Panda Bear vs Australia which informed me "Australians value authenticity, sincerity, and modesty. Giant pandas are solitary and peaceful, but will fight back if escape is impossible. "
Personally, I'm excited about more generative AI being added to my search results, and I'll probably switch to whichever search engine ends up with the best version of it.
And of course the first image for "baby peacock" is the same white chick thing… obviously because this story is making the rounds —_—
Search results that are full of content mills serving pre-genned content: no thanks. It's in the same category as those fake stackoverflow scrape sites.
There is plenty of space there for more volunteer editors to verify content, and likewise, WMF operates its own cloud platform where developers are automating tools that do maintenance and transformation on the human-contributed content.
Then, there is Wikidata, a machine-readable Wiki. Many other projects draw data from here, so that it can be localized and presented appropriately. Yet, its UI and SPARQL language are accessible to ordinary users, so have fun verifying the content there, too!
Not only that, we desperately need cryptographic prof that content X was produced by person Y.
Since most people don't write anything on the internet, I can pay people $5 to use their signature and operate a "sign farm".
Look at the effort these people go though to send their spam.
I’m hoping for a “this human being validated this message”. I think that alone could solve multiple problems.
If you put your name on AI spam people will flag your post and no-one will bother to see your posts for at least a few years.
I wrote some ideas up about this many years ago: https://github.com/pjlsergeant/multimedia-trust-and-certific...
I'm talking about those 300-word, ad-ridden crap articles that are SEO'd right to the top, and if you're lucky you might get the 3-word answer you were looking for: "<300 words of shit>... and in conclusion, <1-step answer>.". Anyway, humans have been getting paid pennies to write those for a while.
AI just turns the throughput on that up to 11, where there's just no end in sight. I think this is like the primary failure mode of AI at this point. It's not going to kill us - we're going to use it to kill the internet. OTOH, maybe then we just go outside and play.
And long before the Internet, people slapped random concoctions together and sold them as medicine, advertising them as cure-alls.