Hacker News Reader

Comment by jron | Hacker News Reader

SEO is crushing the utility of Google. It is pretty telling when you need to add things like site:reddit.com to get anything of value. Harnessing real user experiences (blogs, etc) is the key to a better search engine. This model unfortunately crumbles under walled gardens which is increasingly the preferred location of user activity.

View on HN

That’s where blogs were at, but now a massive portion of them are content farms / splogs.

You’re right that the walled gardens have hurt this. So often I search something specific, or a topic, and find very little. But I know there are communities on Facebook for this, I know there would be peoples posts out there on Instagram which 100% answer my question. But they may as well not exist. Unless I was “following” then when it was said, and mentally indexed it, these things are mostly unfindable, and that’s if I even have an account for said service (which I don’t for Facebook)

It’s sad, more people than ever using the internet, more content & knowledge being created than ever before, yet it’s no longer possible to find the great answers.

Combined with disinformation campaigns and post-truth phenomena, what we defined as "the information age" seems to have been short lived.

> Harnessing real user experiences (blogs, etc)

This is what we need more than anything. More independent blogs. The ability to search events now, or 10 years ago, mass indexing of RSS feeds, etc.

A general search engine is kinda way out of the ballpark for now. But you could specialize for long form blogs, from all sides, hard-left, hard-right, women in tech, white supremacists, all the extremes and moderates.

I've love to have an interface to search a topic and see what all kinds of people have posted long form, without commentary or Twitter/Facebook bullshit "Fact checking" notices. I what to see what real writers are seeing across the spectrum on a given topic for the week or month.

Its hard to get readership writing blogs these days. Thats pretty demotivating.

Also difficult to distinguish a blog from a content farm if you are just crawling the web. Any content pattern you select for would likely be quickly adopted by SEOs.

You could use machine learning instead of a hard-coded heuristic.

I've found a direct correlation between the chance of a content farm and the number of ads on the blog. With 0 ads, the likelyhook of a content farm is 0%.

> This is what we need more than anything. More independent blogs. The ability to search events now, or 10 years ago, mass indexing of RSS feeds, etc.

Thought experiment: what would a search engine look like if it only indexed RSS and Atom feeds?

> More independent blogs.

The problem is that content farms have mastered the art of writing like an ostensibly independent blog. This is most visible in recipe blogs, where for example the site will look independent, the blog owner’s "About Me" page will say that she is a young woman born and raised in Louisiana and passionate about her home region’s cooking, but the English is replete with the sort of mistakes that non-native speakers make. You can tell that the content writing was farmed out to someone from Eastern Europe or Southeast Asia, and basically the whole blog and its owner are fake. (Even all the recipes were drawn from other blogs, but someone was paid to rewrite them slightly.)

I wonder how viable it would be to just exclude all sites with ads.

Reply on news.ycombinator.com