SearXNG in Rust | Hacker News Reader

SearXNG in Rust

github.com

76 pointsdluuuu24 comments

Hi everyone,

I am the original author of Searx. I'm working on a new search project, Hister with a similar motivation: reducing our dependence on external search engines while keeping searches and personal data under our control.

Searx is a metasearch engine that forwards queries to other search providers. Hister takes a different approach. It builds a private full text index from content you choose, then searches that index entirely on your own infrastructure.

Hister can automatically index pages through its Firefox and Chrome extensions. It can also watch local directories, import browser history and bookmarks, index individual URLs, and crawl complete documentation sites.

The feature I find most useful is offline previews. Hister stores the readable content and HTML of indexed pages locally. You can open a result in a clean and sanitized preview beside the search results without visiting the original website again.

Some other features:

- Full text search across web pages, PDFs, docx files, Markdown, OrgMode and text files

- Phrase searches, field filters, date filters, wildcards, negation, aliases, labels, facets, and result priorities

- Optional semantic search using an embeddings endpoint you configure

- Persistent website crawls

- Imports from browser history, Linkwarden, Karakeep, Shaarli, Wallabag, and Linkding

- Web, terminal, command line, HTTP API, and MCP interfaces

- SQLite and PostgreSQL support, plus optional multiple user hosting

Hister cannot replace a global search engine (yet) for subjects you have never encountered because it only searches what you have indexed. My workflow is to search Hister first, then use its shortcut to fall back to traditional search like Searx when I need broader web results.

The project is free software under the AGPLv3+ license. It can be installed as a standalone binary or with Docker.

Project: https://github.com/asciimoo/hister

Website and documentation: https://hister.org/

Small read-only demo: https://demo.hister.org/

---

Btw, this rust implementation looks clean at first glance. The engine interface is simple enough to allow rapid engine development - which is probably the most important factor in a metasearch project.

Hister was one of my main inspirations in starting this project. Great to have your feedbacks :)

Keep up the good work. If you allow a suggestion: implement an xpath or css selector engine (check searx as a reference), so you can quickly integrate quite a few search engines directly from the default searx config.

These are both very nice projects! Ive unfortunately learned too late with hister that the browser bookmark ingestion expects the binary to be local, instead of on a server/vm. But other than that i'm glad these two projects seem like they complement each other.

You can configure the server URL for the extension by clicking on the cogwheel icon on the extension popup or by visiting the extension's settings page. Servers/VMs are fully supported even with user handling.

ah apologies, I thought bookmark ingestion wasn't a part of the browser extension. I'll check there.

Oh, you mean the bookmark import? You can specify the Hister server with the -u/--server-url flags from the command line. Check hister --help for more details

Is Hister able to index Paperless-NGX documents? I’m guessing it works on an Obsidian vault since it can search markdown docs?

Is there any plan to incorporate the same kind of searching functionality that searx has? i.e. fan out to different engines/indexes?

I’ve always been tempted to set up SearXNG, but all the search engines already make me do captchas and anti-bot checkboxes every few searches. Wouldn’t a meta search engine run into the same issues? Or if I had a local server scraping search results pages wouldn’t I instantly get banned from all the major searches? I guess I already feel pseudo-banned from them so maybe it’s not actually a problem.

Just a few minutes ago I dared to do a google search and Google refused to show me results because apparently my iPhone with latest iOS seems like an “automated search”.

On the off-chance that you were searching in incognito (that's where I run into that the most), iOS now lets you pick a different search engine in incognito mode. Personally I use DDG there.

i dunno how searxng avoids it but i've been using it regularly lately with a custom agent, and i've never had a blocked search

No, they do a pretty good job at getting around it. I only run into issues with Brave/Startpage, Google/DuckDuckGo/Wikipedia all work fine. And when you're fusing together multiple sources getting CAPTCHA'd is not actually a big issue, it's more so a thing to fix later.

I would say also -- if search engines are making you do CAPTCHAs every few searches, this is probably an issue with your exit node. I notice a sizable increase in CAPTCHA barriers when I'm on my VPN (most annoyingly Google/YouTube, which sometimes will serve you an infinite number of CAPTCHAs w/o letting you through).

By contrast, my SearXNG instance is making ~100 searches a day from a known good IP address (being my VPS), so rate limiting is less of a problem. You can also set up API keys for many search engines.

I wish it was a problem with my vpn/exit node. But I get these captchas when I’m not on a VPN. I assume it’s just because I don’t login to Google and I enable tracking prevention stuff in Firefox? But that’s really only on my desktop. I get them on my iphone too which is vanilla iOS safari. I get captchas and the cloudflare checkboxes extremely frequently, and sometimes cannot get past them at all.

I saw the articles about residential proxies on TVs recently and honestly have been worried something like that was the cause, but I’ve kept an eye on my network traffic and I don’t notice anything suspicious. Pretty sure I’m not part of a botnet or proxy or anything; but maybe other nearby IPs are.

Maybe I’ll give SearxNG a try. Searching is bad enough as it is, it can’t make it too much worse.

What 's worth to mention is that no sorrowful effortless awful LLM/"AI" use is seen to be used, from the first glance, thankfully...

- https://github.com/MikeLuu99/searxng-rust/graphs/contributor... (as of 2026-08-03)

Though without actually checking out the source code, I believe it initially suggests respect, trust, and heartfelt gratitude towards the author, their experience, and the attitude the author bases on their miracle to be inscribed in the infinite history we all participate in...

Thank you, dear MikeLuu99, for you actually developing, being an actual developer, and the art... you do...

I wish you safety, prosperity, stability... and to your project to be acknowledged, receive enough attention, and to successfully set itself as a miracle to cherish...

I'm not sure how much we can trust software that hasn't been security audited by Ai going forward. Humans are not that good at writing code, we are going to need help that relentlessly looks for issues. We should certainly avoid being religious about such matters, it's engineering afterall.

Thank you. Good point, but I did not mention anything about "auditing" but: "developing", "accountability", "art", and "mind".

There's no "religion" nor "subjectivity" here, and I have no issues with any developer who uses LLM for reviewing and auditing if they have no time or don't know anyone nor participate in Communities who will support a project in the subjects required.

If you still develop with your mind, accoutnability, creativity, and effort towards actually learning, getting valuable experience, and always prioritize a human, art, and effort, you should better know when use an LLM and when not, considering an undefined magnitude of marvelous knowledge, people, and experience you will definitely miss meanwhile, possibly remembering nothing and have no idea what you're doing at all alongside the non-accountable and prone to nonsense algorithm as LLM.

For example, if you do not consider the meat-ground miracles and artworks inside a yet another "free"/sold LLM dataset of now defaced/unknown artists, developers, engineers mentioned, people (like you and me), then if you prompt for a case to audit, require all sources to the evidence/suggestions found, carefully read the output, and manually approach it making notes to learn from and base your future on, it still may be considered an accountable development process.

Cool project! The title is not quite accurate though, the project is "a metadata search engine similar to searxng in rust". It doesn't look like it implements / intends to implement any of SearXNG's advanced features, like weighting of individual sites in results, wikidata integration, or... a frontend :-)

It's not a metadata search engine, it's "meta" search engine.

Nice signal for lack of dominant AI use, it seems :D

I actually thought it was "metadata" all along since it fetch specific fields from the websites' metadata.

Then it wouldn't be "SearXNG in Rust", would it?

Please consider adding a Docker image option

I was just looking for something like this as I didn't want to embed the original SearXNG as a Python package.

you can use SearXNG with an API call, here is an example

https://github.com/verdverm/gmd/blob/main/pkg/web/providers/...

Reply on news.ycombinator.com