irevoire

LMDB-based database for storing and retrieving geojson within Meilisearch

> But in this case, when you opened the GitHub issue, we noticed that you’re part of the Meilisearch team, so we didn’t want to spend too much time explaining something in-depth to someone who was just doing competitive research, when we could have instead spent that time helping other Typesense users. Which is why the response to you might have seemed brief.

Well, in this case I was just trying to be a normal user that want the best relevancy possible and couldn’t find a solution. But the reason why I couldn’t find it was not because you didn’t want to spend more time on my case, it was because typesense provide no solution to this problem.

> it’s a well researched topic that has a bunch of academic papers published on it. So it’s best to read those papers to understand the tradeoffs involved.

Yeah, cool or in other word « it’s bad, we know it and we can’t help you, but it’s the state of the art, you should instruct yourself ». But guess what, meilisearch may need some fine-tuning around your model etc, but in the end it gives you the tool to make a proper hybrid search that knows the quality of the results before mixing them.

If other people want to see the original issue: https://github.com/typesense/typesense/issues/1964

Are you sure it really limits the RAM? You’re still using the same kernel, and if a process is using more memory than another one I would expect the kernel to keep more of its memory page in RAM than the other.

What was your strategy to measure that?

I hate the way typesense are doing their « hybrid search ». It’s called fusion search and the idea is that you have no idea of how well the semantic and full text search are being doing, so you’re going to randomly mix them together without looking at all at the results both searches are returning.

I tried to explain them in an issue that in this state it was pretty much useless because you would always have one or the other search strategy that would give you awful results, but they basically said « some other engine are doing that as well so we won’t try to improve it » + a ton a justification instead of just admitting that this strategy is bad.

Hey, as the previous person said you cannot really limit the memory, Meilisearch uses.

But your OS will share the Meilisearch memory with other process seamlessly, you don’t have anything to do. In htop it’s the yellow bar, and it’s basically a big cache shared between all processes.

AI is completely opt-in. As long as you don’t specify an embedder in the settings, you can continue using the engine as usual. There was no breaking change and the performances improved a lot in v1.12 so you might want to give it a try!

Nice results. I wrote the first version of the relevancy benchmarks

Can't wait to see the same benchmarks against Typesense or other engine closer to Meilisearch

And being fast at writing bugs should not be considered as being productive. I think that's the kind of stuff google is taking into account when they say that rust is worth it.