Wiby.me: curated search engine for content-first suckless sites | Hacker News Reader

Wiby.me: curated search engine for content-first suckless sites

wiby.me

136 pointsnateb202265 comments

The site's definition of "content-first suckless" is more or less synonymous with "created before 2005". Which is fine if you are into that aesthetic, but it is a terrible way to judge the value of the content itself. A few lines of CSS to set sensible margins, spacing and text contrast isn't the end of the world. Heck even Wikipedia is not "content-first" enough to be included in the results. And it doubly sucks for those who are expecting the tiniest bit of accessibility on the internet.

Plain HTML typically has great accessibility. Its when you start adding CSS and scripts it worsens.

Try using a screen reader on a website that uses HTML tables for layout and you will quickly realize how wrong that statement is.

Eh, can work pretty well compared to CSS. Table cells are always in the order they appear on screen, and if you read them in sequence the order makes sense. That does not apply with CSS based designs.

OK, but go try a screen reader on a sampling of both kinds of sites and you will quickly find out that what you think is true is wrong.

What damage does CSS do to screen readers and accessiblity? I've used several accessible browsers it has never bothered any of them.

CSS can completely change the position and semantics of a tag.

Floats, z-index and position:absolute/relative are especially tricky.

> Heck even Wikipedia is not "content-first" enough to be included in the results.

try searching “wikipedia” on wiby before claiming this next time, thanks.

What set of Wikipedia articles are indexed by this engine?

if i understand it right, whichever Wikipedia articles users have submitted will (usually) be indexed. i don’t know that there’s an API to query the index set, or not. i submitted the Wikipedia article on The Experience Machine an hour ago, so i’m watching to see if that’s approved (it’s not instant). given that the Wikipedia article on PAL is indexed (i see it when i search “wikipedia”), i expect most Wikipedia articles are acceptable and it’s just a matter of which ones people (like you) will have submitted.

tangential to your question, i think it’s worth pondering what makes (or would make) a per-page moderated/curated search engine like Wiby valuable, though.

Wikipedia articles are easy to find. they’re all generally good quality. you don’t really need a new search engine to make Wikipedia more accessible.

Personal websites, on the other hand, are a mixed bag. there’s a tremendous amount of worthwhile stuff out there, where one page might explain how to solve a niche technical issue and the adjacent page might just be the author sharing photographs of a recent hike, or their wedding, etc. currently, both of these pages tend to be low-ranked on most search engines. i believe one prospect for Wiby is to bring the former to prominence.

it’s definitely not a general-purpose search engine like Google. but try searching “raspberry pi” or “home automation”. on Google you will get pages focused narrowly on products: where to buy rpi’s, a dozen different home automation brand webpages, etc. on Wiby you get blogs documenting things people built related to rpi or home automation.

it could be something like a search-based, topic-agnostic Hackaday.

it’s not perfect. it only has a few relevant results for each topic. but there are certain specific types of results which are near impossible to find on Google but can be found on Wiby. maybe that can be valuable. maybe it will fade way. i won’t forecast the future, but i am intrigued.

Seems really strange that this would work on a per site not a per domain basis.

If you want Wikipedia just search Wikipedia. I don’t need the first page of results filled with sites I already know about. Why does Wikipedia even need to be indexed at this point?

There is a threshold for how much css/js gets loaded. Exceptions made for side wigits on a page, and web forums.

I've found myself confused by this:

https://en.wikipedia.org/w/load.php?lang=en&modules=ext.cite...

On a side note why does every wiki site have something like this? I can't tell what it is and haven't had time to look into it. But thats more than just a few lines of js to layout a page.

Lastly, some sites like Wikipedia have hundreds of millions of pages, allowing them in would swamp the index as some people cant help but want to submit hundreds or thousands of pages from the same site (which would get blocked). I don't have enough resources to carry the zillions of wiki pages and have them drown out everything else. Most people know where to go if they want to look something up in Wikipedia and Google does a great job of it for just about every query you make, there will be Wikipedia prominently up there as a top result.

There was a time, getting to be quite a long time ago now, when you'd search for something - software to run on your old SGI, pictures of your 1982 ERA Vans, etc. etc. - and you'd find it. You'd find it because someone's 'thing' was old shoes, and they'd built a web shrine to their thing. These were people who had knowledge of things unrelated to HTML/HTTP/CSS and so the sites were rudimentary.

You are right, the engine should focus more on content and less on the complexity or quality of the formatting - but I also kind of get it.

First two things I found with “Surprise Me”:

Computer Closet: http://www.computercloset.org/compindex.htm

Guide to Spam-like products: http://spam.budwin.net/

Already a fan.

I’m sure I could find something like this in Google if I tried hard enough, but damn, you just don’t see this kind of stuff in Google anymore (or at least I don’t).

EDIT: here’s another.

https://www.mrbreakfast.com/

This is fantastic! Based off some of the other comments I don’t know if this will be a quality search engine, but I think this might be the second coming of StumbleUpon.

> Computer Closet: http://www.computercloset.org/compindex.htm

View source ...

:-)

> https://www.mrbreakfast.com/

Surf's up! That's 100% pure goddam information superhighway right there.

This is apparently FOSS under the GPLv2: https://github.com/wibyweb/wiby/

FWIW I don't see any mention of suckless on Wiby itself, so the headline may be misleading. As someone who has a negative impression of the suckless community, despite agreeing with their goals of "keeping things simple, minimal and usable", this matters.

This is great. I would love to see engines of custom curated content, like the readsomethinginteresting.com blogs or academic articles.

Suckless for me stands primary for the idea, only secondary for the suckless.org crowd.

The creator's description of the types of sites indexed (from the Submit page[0]):

> What kind of pages get indexed?

> Pages must be simple in design. Simple HTML, non-commerical sites are preferred.

> Pages should not use much scripts/css for cosmetic effect. Some might squeak through.

> Don't use ads that are intrusive (such as ads that appear overtop of content).

> Don't submit a page which serves primarily as a portal to other bloated websites.

> If you submit a blog, submit a few of your articles, not your main feed.

> If your page does not contain any text or uses frames, ensure a meta description tag is added.

> Only the page you submit will be crawled.

Some additional ethos info is on the About page[1].

[0] https://wiby.me/submit/

[1] https://wiby.me/about/

It's kind of funny how many commenters expect all results filtered to their liking. Most often when the topic comes up, the crowd often denounces Google et al preselecting content. Can't have your cake and eat it, too.

Have you ever tried scrolling over things you don't like?

We need a search engine that prioritizes ad free sites over ads. The less ad a site has, the higher it will be ranked. And you will automatically get rid of all search spam following this one rule.

This is how you end up with a search engine for pay-walled sites

Or how you end up with a search engine for passion projects, open-source projects, community hosted sites and more.

Treat paywalls the same as ads.

How could the crawler discover the pay-walled content? If sites explicitly allow the crawler to enter, what would be their argument to prevent that the search engine shows a cached version, especially if that's the search engine's policy from the start?

That said, search engine for pay-walled sites with integrated micro payment isn't a bad outcome. What else could create an alternative to the ad-based business model?

Most are are now affiliate links built into the content.

Well, I just popped an inocuous search term ("dinosaurs") in there and the very first result was a page with the title:

"Evidence That Humans And Dinosaurs Coexisted"

So that actually kind of sucks.

I thought it might be a lovingly handcrafted artisinal static site at least, but it's very very 90s web design.

don't see how "lovingly handcrafted artisinal" and "90s web design" are incompatible. ;)

the thing about artisans, is that they create according to their specific taste, which may or may not intersect with subjective modern tastes. :)

A lot of new wave personal websites are using the 1990s as inspiration, due to the 1990s having far more web design versatility than the utter boring designs that every copies nowadays.

this is absolutely amazing

i love it

But on the same page of results you also get things like this: http://mlwi.magix.net/synchronicity.htm

I don't see what sucks - we're talking about the WWW here, and we should expect the full spectrum of thoughts and opinions. Unless you're hoping that each and every boutique search engine will filter the results according to your preference?

When ordering results, one would expect the top results to contain information mostly pertaining to the search term.

"Dinosaurs" and "my old book says the world is only a few thousand years old, therefore I'm going to try to convince you that carbon dating is invalid by hand-waving my way to the conclusion that Dinosaurs and humans existed at the same time" is a bit of a stretch. I'd expect maybe "Triceratops" as a result near the top, for example.

This isn't a search engine that is optimized for quality or authenticity of information contained on the sites. It's simply a tool for exploring an index of websites that follow the guidelines set by the author.

I'm not talking about quality or authenticity. Literally just the topic.

Hey, you asked for dinosaurs and it gave you dinosaurs. A simple keyword search possibly, with pages that have more 'dinosaur's in them ranking higher

Well, it's a webpage exclusively devoted to dinosaurs. Dunno what else to say.

The engine is not trying to push a political agenda. It found a result with the word dinosaur in the title. Calm down.

You might be reading into my sarcasm a bit too much. I'm saying the links aren't relevant. That's it.

As to your demand that I calm down, again, you're trying too hard.

I think the content not sucking is orthogonal to the design of the page not sucking.

I took this as a cue and searched some loaded terms.

"vaccines" and "covid 19" had decent results, mostly people's blogs and some sites trying to prevent misinformation.

"abortions" mostly gave statistics, a blog by a 14-year old student in China, and an extensive collection of writings in support of Ayn Rand and against Noam Chomsky.

"trump" gives the most perplexing results, which I don't really know how to describe. Interestingly, the Rand/Chomsky page shows up here too.

I suppose it depends on your definition of suckless, but all the sites that were returned were lightweight, and content-first. As for the content... interesting might be the word I'd use. And it definitely doesn't show up in mainstream engines.

Why does it kind of suck? Let people believe what they want to believe. It's not like it makes any difference to your life or theirs.

Wiby is a great example of how welll a curated index cam perform.

I've been toying with the idea of like crowd sourced index curation. Like maybe backed off a git repo or something. Would be an interesting experiment.

I entered "hacker" and found this: https://wh0rd.ca/humor/text/Adequacy_org%20%20Is%20Your%20So...

A classic. I was very much that kid in the mid 90s too :)

Lunix indeed.

omg, that's awesome, lol

Falls foul of the old 'content stuffing' tricks where you'd have random text in white to fool search engines into showing your page for unrelated searches - I searched for 'projection TV' and got ape-o-matic.com which didn't feature projection TV at all... Except in the key word stuffed part of the page

QUOTE

r6000 1 real estate laws 1 "Kitty Hawk Inc." software 1 Mt. Mansfield 1 jake OR rothermel 1 obsession 1 REALTOR.COM REAL SELECT 1 black jack 1 micheal jackson hate 1 repair projection tv screen 1 v tech 1 german meats 1 compare automobile 1 varmvattenberedare 1 Free Family tree information 1 digsolve.zip 1 epson lx-800 dip switch 1 télévision 1 offf -road 1 "michael robertson" 1 flight attendants tower air 1 "westwood studios" 1 aled jones shirtless 1 boomtown 1 hampton court 1 pics of power rangers nude 1 linux fvwm 1 pissing pisses femal 1 Fresh water aquarium 1 file compare programm 1 1979 4x4 chevy trucks 1 beanie babie magic 1 Cheyenne Frontier Days 1 apedemak 1 pennies 1 rap hip hop ra 1 "loving every minute" 1 sat-a2 1 45 auto 1 shack map 1 micro dc30 1 providian 1 ski conditions 1 adult key sites 1 people finder telephone numbers 1 icebreaker 1 WWW.SONY.COM 1 WICKED WOMAN 1 bank-owned reo orange county 1 university maryland 1 teen hideout 1 canyon texas real estate 1 script of scream 1 Last-Minute OR orlando OR NurFlug 1 "eternal champions" 1 "barrel house" 1 midi files about slayer 1 xmas cards 1 flashnet 1 Judge Roy S. Moore 1 beautiful latino women 1 "Stacy Sanchez" 1 NASCAR Dale Earnhart 1 "mac warez" 1 puritan pride herbs 1 "

Searching for "metaverse":

- Mark Rosenfelder's Metaverse -- Bob's Reviews Women in Comics Zompist Phrasebook

- Metaverse Christianity and the Problem of Shame

- Piero Scaruffi's knowledge base -- Singularity Metaverse Blockchain Virtual Reality A Timeline of Artificial Intelligence A.I. slides Future of Technology Tributes Birthdays: a secular calendar of saints Centennial

The "sites which suck" filter seems to have left only sites of random blithering. After eliminating ad-heavy sites and the usual suspects, there's no easy way to rank. Now you need real content evaluation. Which is really hard.

You'd probably get better results by searching maybe 50 sites, such as Wikipedia and Brittanica for general knowledge, and a few just-the-facts new sites (Reuters, BBC, Japan Times, the Economist, the Guardian.) omitting their opinion articles. For popular culture, go for the trades - Variety, Chartbeat, etc.

A search engine that's just Wikipedia and Reuters sounds like the most boring website on the planet. Sometimes people don't want "just the facts," all the time. Sometimes they just want to have fun. And obviously "metaverse" won't yield many results in a 90s-style-website search engine.

Maybe im biased here.

But if wikipedia (which doesnt seem included) doesn't count as a content-first site, i dont know what does.

Yeah, the first result of the "surprise me" search is an early-2000s era style 9/11 conspiracy site. Brings back memories, but not exactly "suckless".

"suckless" often means for websites; loads quickly and does not overuse JavaScript.

I like the old web. And I love the old web revival. Been digging into wiby for many hours at the point the same websites appear more than once.

Works perfectly in w3m over Tor and returns useful and interesting results in a fraction of a second. Bravo!

Renders poorly on an iPhone (way too small).

Searched for wow, got a lot of fishing sites. That was surprising.

Well most results are desktop sites. Mobile users probably aren't the intended audience.

It's so refreshing to be able to tab through search results without hitting a thousand irrelevant images or ads or menu items.

I searched NixOS and found really useful blog-articles, the kind you would find submitted to HN. Quite cool.

Talk about synchronicity, earlier today I was thinking that someone like this ought to exist, and here it is!

Thanks for sharing.

Zero results for requests in japanese, russian or hebrew. Suckless they say? What content?

I love this. I searched for guitar and every result felt like a geocities site circa 1997.

i remember when this site came out and the author posted it on 8chan /tech/.

Reply on news.ycombinator.com