This is a structural problem and anything that gets large enough will succumb to the same forces. If the incentives are for optimizing ad revenue then that's what all corporate machines will do at scale, regardless of their initial motives and incentive structure. It doesn't help that Google is also an ad network, hence the ouroboros aspect.
It's the same with social networks, upvotes and likes skew the the type of content that is generated to be liked by the people liking and upvoting instead of being insightful. Popularity is not the same thing as insight so the social web is full of mostly useless but popular content.
Paul Graham has been pushing for someone to disrupt web search on and off for at least the last 10 years: http://paulgraham.com/ambitious.html
(he tweeted about it more recently, but I couldn't dig it up since I'm not much of a tweet man)
Google made links on the web the measure of how good a page was. That became the target of everyone trying to do SEO. As a result, it stopped being a good measure of how good a page was.
But in the long run, nothing will work in that environment, because every measure will be gamed as soon as people figure out that Google is using it. Google's only choice is to try to stay ahead of the SEO crowd, and I'm not sure they can do that (well) for too much longer. In fact, if the article is to be believed, they're already starting to fail.
This is also the case for social media platforms. They're incentivized to surface content that generates engagement and ad revenue. Basically ads are at the root of all problems when it comes to the internet and the content on it.
Uh, yes it is. If the owner of the site being searched is generating profit from that site being searched then they will game the search engine's algorithm to get them the most clicks.
At the moment Google is incentivized to uprank spam because the spam comes with ads from its own ad network.
A search engine finds 50 pages that are exact matches for the search. Which one does it present as the top of the list? How does it decide? Unless it decides literally by a random number generator, however it decides, someone will try to discover the algorithm, and exploit it. This is true whether or not the search engine allows or displays ads.
> I'm not interested in a pedantic argument. If you didn't understand what I meant then you should have asked for clarification.
I don't think it was a pedantic argument or a misunderstanding. I think Lascaille understood your position and disagreed with you, not just pedantically but over the substance.
Popularity is gameable but not the same way as Google is currently gameable because as soon as a site becomes popular and starts exploiting its ranking it will be easy enough to add a decay factor to prevent such sites from dominating the top results during the multi-armed bandit stage of ranking.
In any case, the logic of why Google is going to shit is obvious. Arguing about fixes is not going to change their underlying business model and why spam is dominating their results. As long as they are a search engine, an ad network, and a corporation that must maximize profits their results will continue to deteriorate until the top results are all just spam.
> I don't think it was a pedantic argument or a misunderstanding. I think Lascaille understood your position and disagreed with you, not just pedantically but over the substance.
Then that wasn't clear and seemed like a pedantic point since it's obvious that any algorithm is gameable and I should have made it clear that I wasn't talking about a perfect search engine but one that was not susceptible to profit driven spam (which is currently the reason that Google results are going to shit).
In my mind, if the web page tries to exploit knowledge of the search engine's algorithm, that's "gaming". This is done by the web page, without the deliberate co-operation of the search engine.
If the search engine is the one doing the funny business, to increase their own profit, to me that's beyond "gaming". That's... "corruption" might be the right word.
Yes, I was a bit disappointed with the reddity response. The sites admin has an incentive to have their site appear first. The quality of their content may not suffer but they will additionally try to identify how the algorithm works and 'game' it.
The problem begins and ends with the conflict of interest that Google both sells ads and selects search results. If they didn't have a vested interest in people visiting sites with their ads on them, they could decimate the number of spam results.
If a site is hosting ads from the Google ad network and is barely above being spam then Google will prioritize it over other results in order to maintain its quarterly revenue predictions.
I use your crazy search engine all the time, and it's fucking great. Last time I checked, though, you're getting like $30 a month from Patreon and that's it, haha.
You're too much of a well-adjusted normal human being, that's your problem. You need that touch of Zuckerwellian sociopathy to get to the next level
To be serious, it's notoriously difficult to actually make a profit off search. I guess we'll see if Kagi will be able to eke out a market for itself, that may change things. Right now, I think investors will look at Bing and go "yikes".
Doesn't go to say I couldn't build a fantastic search engine with more resources than my current shoestring budget, the hard question remains: Why isn't this just a waste of money? Would take one hell of a sales pitch to gloss over that concern.
The only thing that influences Google search results is Google's desire to keep as many people using Search as often as possible, since nearly all of Google's money comes from showing those text ads at the top of the Search results. This is all public information, you can read it in the 10K etc.
So if Search sucks, it's not because Google has the wrong incentives but because they can't solve the problems Search faces.
Admittedly more in some areas than others.
If there were a better solution we’d all already be using it. Certainly you can rely on savvy people to produce free stuff, but the total amount of content will be drastically lower and therefore fewer consumers.
The best thing is to just use bookmarks and your favorite sites’ own search.
There’s just too much trash on the net
I'm really interested in the idea of decentralized search where everyone has the power to choose for themselves who to trust.
> How would the sites keep the lights on without ads?
Making them turn off the lights is the goal. Good riddance I say, once we get there.
I wish there was some middle ground. Think of all the useful Youtube videos showing how to play an instrument, do woodworking projects, fix cars... there is a vast amount of knowledge there. Maybe YT should be nationalized :-)
Do you think there's a lot of good content out there left to index?
It's true that walled gardens have been eating up useful information and that is a real shame, but make no mistake: there's still a ridiculous amount of good stuff on the open web. They're not playing the SEO optimization game so they get buried.
I have sort of got stuck in this bubble of reddit and not much else and... the way people behave has really sapped my desire to contribute anything to the world. I'm looking around me at the opinions a lot of people are expressing and I think... no, these people don't deserve nice things. It seems sometimes like the whole world is full of venal, bitter and divisive people that do nothing but snap bitterly at the heels of anyone that makes something of themselves or strays too far from safe consensus.
It also seems to me that internet has somehow became a middle man and is not providing human deep enough interactions, especially outside chat-like website (basically any exchange, business)..
I could envision a whatsapp like system with quality control for producers and transparent transaction/tracking/accounting management offered by the network so people spend less time on side-loads and just focus into helping each others and doing what they need to.
It's a new internet protocol (NOT www) designed to be minimalist and interesting to hobbyists.
> How would the sites keep the lights on without ads?
The same way they did in the web 1.0 days - somebody would maintain the server themselves, or pay to have it maintained.
Discussion: https://news.ycombinator.com/item?id=30072085