Marginalia Search: 1 Year | Hacker News Reader

Marginalia Search: 1 Year

memex.marginalia.nu

140 pointsgeorgehill29 comments

It's one of my smorgasbord search engines run from a script (which collectively do so much better than Google or any of the legacy "big-tech" engines)

Why it matters

One of the great American philosophical orators, Rick Roderick (a Texan cowboy philosopher and academic twin of Bill Hicks), spoke about the importance of "the margins":

Rick notices that in history certain people do not get a voice;

Women. The Poor. Old and young people. The "mad" and disabled. Race minorities within any group. Thinkers more than a decade ahead of their time.

There's even a thing called "marginal analysis" in hermeneutics - not the economic concept - the analysis of things that have been left out of history, and why, and by whom (victors writing and re-writing history etc). Erasure leaves a trace that tells its own tale.

These things have become the grist of social justice today (with all it's positive and negative sides). But "injustice" is not really Roderick's point. It hardly needs mentioning that most people get a bad deal, don't get heard, and their lives fade into obscure irrelevance.

What matters is the enormous value and importance of marginal contributions. These are lost. Almost every great discovery, it turns out on deeper analysis, was made by an (or often, simultaneously several) obscure nobody years earlier, who didn't provide a recognised proof in the right journal, or didn't patent the thing. In Plato and Shakespeare, Roderick says, "It is always the fool who delivers the important news". Things don't start at the center, they begin on the margins and diffuse in until they find legitimacy in an acceptable voice.

A marginal search engine engine is, almost by definition, acting counter to the status quo, as the Internet Archive stands in opposition to fickle entropy of bitrot and ephemeral culture.

Well done on this project and good luck.

You have a neat way with words. I enjoyed reading this.

I'd like to second this and ask how you learned to write?

drBonkers I think most writers get there by reading a lot and we're recurrences of our influences. Content wise I've found mixing classics with modern, deep with superficial, serious with humour, is important. It balances things like a decent exercise regime.

May I ask what other search engines you use? I'm trying to find good alternatives to the usual right now.

#alias search-1='w3m https://duckduckgo.com' alias search-1="torw3m -no-proxy duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/" alias search-4='www https://search.marginalia.nu/' alias search-2='www https://infotiger.com/' alias search-3='www https://www.qwant.com/' alias search-5='www https://rightdao.com/' alias search-6='www https://www.mojeek.com/'

Thanks! That's really useful.

A few more I found: https://seirdy.one/2021/03/10/search-engines-with-own-indexe...

I recommend Teclis, Wiby, Kagi's "non-commercial" lens (mostly powered by Teclis), and and Gnomit (old results only, helps against the recency bias) to add some serendipity to your searches.

I'd be very interested in checking out your SE script.

There's two I use. Nothing fancy like collecting and sorting results from multiple engines. Just a bash sequence.

alias se1="se1.com" alias se2="se2.com" www-browser se1 $1; www-browser se2 $1 ...

Arrange them in roughly the order best to worse. If a result isn't satisfactory I hit q (exit for my choice of browser) and it falls through to the next search engine.

The other is surf-raw (actually written by everyone's favourite political prisoner, Mr. Assange) You can change the "elvi" to be what you want, but imho it doesn't really offer much extra because you still need to remember the aliases you gave each search engine. A sequence in an alias file hides that from you so I just type

$ search "keywords about thing"

Of course it falls over when different search engines parse the urlstring in different ways.

Would you recommend more resources to learn about his work?

Roderick? There's a lot here [1] and the usual life summary here [2]. Sadly he passed away too soon and recordings of his lectures are a bit collectable.

[1] http://rickroderick.org/

[2] https://en.wikipedia.org/wiki/Rick_Roderick

Make sure you use the API if you want to query my search engine programmatically, as I do use some bot mitigation strategies for the web interface. If you want to use the API for more than just one-off queries, you can also send me an email and get a key so you get your own rate limit.

https://api.marginalia.nu/

While this is all true and all good, “marginalia” is a completely different word for a completely different thing. It refers to handwritten content in the margins of books.

The word is a Latin adjective and just means marginal. It's most frequently used for marginal notes, but I don't see why you couldn't use it in other contexts as well.

Not really sure why this feverish ramble deserves the front page, but sure, I'll take it.

First reaction: Wow! That’s an insanely harsh reaction to someone celebrating a 1yr project anni.

Thankfully, I checked poster name. Grats!

I appreciate those who put the hacker back in Hacker News.

Thanks so much for your project! It’s one of the things I turn to when looking to uncover actual human writing these days.

I'll admit, I automatically upvote anything Marginalia-related. We just like you, man! Keep fighting the good fight <3

Show HN: Marginalia – Exploration Mode - https://news.ycombinator.com/item?id=30047455 - Jan 2022 (53 comments)

A search engine that favors text-heavy sites and punishes modern web design - https://news.ycombinator.com/item?id=28550764 - Sept 2021 (717 comments)

I really like this search engine. Particularly when I want to jump face-first into researching a new topic that I know almost nothing about, the signal-to-noise ratio is often so much better than Google.

Honestly it's not working quite as I want it to right at this moment, it's gotten a bit slow due to a thing I'm testing, and it feels like it's not always finding the best results. Still works decently but still a bit wonkier than usual right now.

Only you, as it's sole (or soul even) developer, could possibly be aware of its current state and result sets and how they may or may not be differing from your ideal vision for it.

For the rest of us, it's a really cool project and I, for one, am glad your efforting it...so thanks!

Apparently, if you omit all the we-got-you-and-lock-you-in-your-ad-induced-information-bubble, you can get quite far with even modest hardware:

https://memex.marginalia.nu/projects/gemini-server.gmi

> I put the machine together mostly for a search engine, Because I didn't want an actual rack making noise and heat in my living room, the server is made out of consumer hardware:

> Ryzen 9 3900X

> 128 Gb RAM

> 4x4 Gb IronWolf ZFS

> A bunch of SSDs index lookups

Some of that is a bit stale, I'm not using ZFS anymore because it eats too much memory, and the SQL database containing metadata is on an Optane 900P, but it's still largely the same machine.

But yeah, turns out, if you mind what you are doing when you write your software, modern consumer hardware can be pretty good.

I also think a lot of the architecture paradigms that are common today have a very high constant cost, which may be the admittance fee for high scalability, but I don't have anything to scale up to anyway so my problem is how to do the most with fixed hardware.

If I had written this like a cloud application, there is just no way it would run on the same machine. I'd need ten times more hardware. I'd also not be anywhere near finished.

Data locality is slept on in terms of performance.

Marginalia is a breath of fresh air in an internet that seems to be...disappearing. I actually like it's philosophy of not doing "useful" search queries - it's great just to see what's lost behind Google's wall of Reddit/Wikipedia/SEO Blogspam results. Funnily, after playing with it for a bit my gripe is that there isn't a way to exclude results from vanity domain names which are consistently the lowest quality results.

Try the blog search profile, and if that's still no good, add set it to deny javascript. That reduces quite a lot of noise.

>I actually like it's philosophy of not doing "useful" search queries

Generalising this a little, it sometimes feels like the world has become all about optimisation and productivity, and the beauty of uselessness has gone missing. Many of my favourite posts/projects on HN are from people who have set out to build something utterly useless in the name of fun - and sometimes they turn out to be useful after all!

Reply on news.ycombinator.com