mfkhalil

This is cool. Love how well it fits in with the site. Will probably keep it on.

Hey, we’re working on MatterRank which is pretty similar to this but currently works on web search. (e.g. I want to prioritize results that talk about X and have Y bias and I want to deprioritize those that are trying to sell me something). Feel free to try it out at https://matterrank.ai

Would also be interested in hearing more about what you’re envisioning for your use case. Are you thinking a browser extension that acts on sites you’re already on, or some sort of shopping aggregator that lets you do this, or something else entirely?

I think in a lot of cases that's because the meta with LLMs right now is to "have them do things for you", which generally means that obfuscating what's actually happening behind the scenes can make them seem "smarter" to the average user. Also, engineers are used to full control over deterministic input-output pipelines, which is a framework they try to force on LLM applications that fails miserably for the reasons you've listed.

In my opinion, the best applications of LLM UX will have full clarity for the end user (something we're trying to do with MatterRank). The non-determinism should be something the user can control to get better results, not something the engineer has prompted that takes control away from the user.

Now, if the use case you're looking for is "give me results with x text", then yes I agree with you that LLMs are just getting in the way. But that's not always the case.

Yeah LLMs were the easiest way to get a proof of context running, but replacing it with a specialized distilled model/classifier should hopefully make it way quicker.

As for the results, it's tough because we've made the deliberate decision to have no control over the reranking. What that means is that if your criteria is "written by a woman", for instance, then any result that meets that will be ranked equally at the top. In all engines I've built for myself, I have a relevance criteria that's weighted relative to how much I care that the result is exactly what I'm looking for. It's probably important to make that clearer to the end user.

Credit to @ziftface — I should’ve included more examples in the original post. MatterRank is useful when you want results with specific qualitative traits that go beyond keyword matching. You can ask for stuff like “written by a woman,” “mentions these specific lines from a movie,” or “talks about X/Y/Z but avoids A/B.” Since it reads the full content, not just metadata or SEO signals, it lets you be a lot more precise in ways that traditional search engines just don’t support.

You don't have to create an account

Yeah that's fair, "doesn't know what we want" might have been oversimplifying. Better phrasing would have been that there is a very hard limit on the context you're able to give when using a search engine. It's mainly keywords, and then maybe some tricks like `site:` or quotes.

> Do you think you're getting more value than 4 iterations on the initial search term?

Definitely not for all cases, but in some cases yes. Where it really makes a difference is when you're looking for qualitative attributes of the webpage, rather than what words show up in it (e.g. “written by a woman", "is likely to convince someone who supports Trump", "talks about X/Y/Z but not A/B.”) It reads the actual content, so you can get oddly niche in a way you just can’t with keywords alone.

Because LLMs understand language, we can start building algorithms that respond to what users say they want. Instead of reverse-engineering user intent from behavior, you can just tell a system “more of X, less of Y” and it listens. Way more flexible than hard-coded workflows.

Actually the tutorial was breaking the homepage so took it down while it was being fixed. Should be back up now.

Fair point, I probably should have provided more context in the post.

MatterRank uses LLMs to rank pages based on criteria you provide it with, not SEO tricks. It’s not meant to replace Google, but helps when you're looking for something specific and don't want to wade through tons of results that you don't care about. Still early, but useful for deeper searches.

MatterRank is pretty slow still since it runs LLM evaluations on each webpage as markdown content. Wouldn't really consider it a Kagi alternative (which I haven't used but have heard great things about!), as that's more of a search engine in the traditional sense.

I think where MatterRank shines right now is for finding results where you wouldn't mind waiting an extra 20-30 seconds for an added layer of vetting, as opposed to just wanting a quick answer.

Having said that, we are definitely working on making it faster and more useful for everyday queries.

If you're referring to sponsored content, then you can actually use MatterRank to configure an engine to devalue content that's trying to sell you something or is sponsored.

If you mean pop-ups, MatterRank can't handle that at the moment because it evaluates markdown content, but it's something we're looking at adding. In the meantime, I'd recommend a good ad-blocker.

I actually completely agree with this. Search is a good example, but in general it seems that general consensus has become that consumers don't know what they want, which is pretty frustrating, and probably a product of the success of the TikTok algorithm and similar software.

I'm hoping that as LLMs become more mainstream more functionality is built into tech that doesn't treat consumers as idiots. This is one stab at it, but there's so many other opportunities imo.

This is awesome!

Zoox (Amazon Subsidiary) should be rolling out soon. They've already done successful test rides and should be launching in Las Vegas next year.

Best I've been able to get on the 4x4 is 78– let me know if you can beat that!

Appreciate all the feedback! Some updates:

1. Just pushed a version with citations and corresponding sources.

2. Also added the ability to search via URL Params & changed the html input field type to "search". Searching with params will work with any of ?s=, ?search=, ?q=, or ?query=.

3. When it comes to sustaining the app, we do have some ideas we've discussed internally, but as of right now we have a small amount of funding which we're relying on. Our main focus at the moment has been building a version of this that we prefer using over existing products. If anyone has ideas or suggestions on how to monetize this outside of the traditional options (Ads, Pro plans...), we'd love to hear it!

4. Regarding open-sourcing, we're planning on keeping the product closed-source for now, but the high-level architecture is nothing too complex and is outlined in the post. If anyone has any specific questions though, we'd be happy to answer them.

5. Moving forward, some of the features we plan on adding are shareable conversations & the ability to branch out and edit older messages.