Hacker News Reader

Comment by marginalia_nu | Hacker News Reader

(Creator here) I recently moved the website from search.marginalia.nu to marginalia-search.com and gave it a bit of a visual touch up, on the basis that I felt it was working too well to be a weird subdomain outgrowth off my blog.

It's still the same search engine :-)

View on HN

Lately there's been some consternation in the small-web community about the aggressive crawling by OpenAI and friends.

How often does yours (re)crawl its indexed sites?

I do a big recrawl around once every 10 weeks, but I have a RSS crawler that does daily visits and fetches new items. But that should be very low volume traffic.

Interesting, does your RSS crawler auto-detect feeds? That's a neat solution I'd never considered.

The big crawler tries to find feeds, and the RSS crawler acts on what the big one has found.

It looks like the "random" button on the old version no longer works. Did you remove that feature?

Hmm, I think it's just a broken redirect. It works if you search for browse:random . I'll try to fix it later today.

Any reason why you don’t use a shorter domain name?

(Amazing what you built, btw)

Hard to find one that makes sense and isn't inscrutable like mxyzp.tlk.

If I ever do find one that's appropriate I might set up a CNAME record to point to marginalia-search.com

FYI - https://www.dynadot.com/market/search has a nice way you can filter their Aftermarket domains for sales by Character Count, TLD, No IDN/Number/Hyphens.

I've found over the years plenty of <= 5 character domains that have a meaningful name just from using their search filters.

(I'm not affiliated in anyway to them)

Reply on news.ycombinator.com