Not yet ready for launch but built a simple demo trying to get into startupschool ( failed unfortunately :( ), which lets you search every hackernews post while letting you filter based on domain / user / story type.
With 50K plus amazing companies that applied, it is very difficult to stand out :( Slowly building a team with some of the best engineers that I had the pleasure of working with to take this to the next level. We badly wanted to get into the startupschool to help guide us and get us to the next level. Wish we were part of the program, but glad everybody gets to view the lectures :)
* the backend you use and how it will scale to sites with large amounts of data across servers
* can third party sites integrate your search service?
* How is it different from eg- Algolia
Good luck with the project!
Backend is custom built written in C and assembly. Supports sharding and replication which is rack aware and data-center aware.
> can third party sites integrate your search service?
Yes of course.. that is the end goal.
Algolia is awesome.. but you end up paying a lot based on how many ways you sort / rank data. This operates with an on-the-fly ranking model and rank on any field in any direction. Also different ranking algorithms, extensibility with Lua and a lot more when I officially do a showhn
You have to write a really long blog post on why you've chosen this way.
Got hooked into it and became obsessed with speed and rewrote everything in 'C'. Replication is based on 'Raft', actually the multi raft variant proposed by the amazing folks at Cockroach (https://www.cockroachlabs.com/blog/scaling-raft/)
It does not use a backend framework. It is a simple http/https server (epoll + multi-threaded) which talks json. I use Jansson for json and utf8proc for unicode handling. Index is custom built.
I have been working on low powered distributed systems for over 10yrs, which certainly helped. Will definitely let you know when I get that blog post written :)
Will probably fix the UI this weekend and do a proper 'Show HN' with more options, charts and analytics.