qdequelen

Hey PSeitz, Meilisearch CEO here. Sorry to hear that you failed to index a low volume of data. When did you last try Meilisearch? We have made significant improvements in the indexing speed. We have a customer with hundreds of gigabytes of raw data on our cloud, and it scales amazingly well. https://x.com/Kerollmops/status/1772575242885484864

> Are LLM tools better or worse than e.g. meilisearch or elasticsearch for searching with snippets over a set of document resources?

Absolutely worse, LLM are not made for it at all.

Thanks for the highlight!

Do you handle high throughput volumes? I would need this for testing to host a database service at scale.

Did you only get the schema.json?

Hey, I'm the CEO of Meilisearch. If your issue is performance, I would love to give you a try with Meilisearch. You'll be able to create an "as you type" experience with our engine that responds in less than 50ms!

Thanks @CaroFG, I'm sure that if anyone has questions, the engineering team would love to answer!

Hello, I'm the Meilisearch CEO. I think you're also correct, Jabo.

I just want to clarify. Meilisearch's pricing doesn't start at 1.2K/month, but at 0/month. We have a usage-based pricing that is basically 0.25/1000 documents and searches. And, funny thing, we are thinking about splitting the searches and documents, too, but we wanted to have more data to be sure to select the right unit price for each. :)

Make the web faster with Rust.

Hey muhammadusman, I'm the Meilisearch's CEO. We have a complete comparison table. Note that it represents our point of view. https://docs.meilisearch.com/learn/what_is_meilisearch/compa...

Both Meilisearch and Typesense are really different regarding resource consumption and performance. I would say that where Typesense would have a better indexing performance (Meilisearch has recently improved indexation speed), Meilisearch will guarantee a much faster search performance while keeping impressive relevancy. Regarding the consumption, as Typesense is entirely on RAM and Meilisearch is using memory mapping, Meilisearch would take more disk space but less RAM.

To answer your question precisely, we handle all the space-separated languages and have specific tokenizers for Chinese, Japanese, Korean, Thai, and Hebrew. We plan to add more languages in the future.

Thanks for your feedback!

You should try it again since we intensively improved the indexation performances. Most of our actual users no longer have performance problems, even on hundreds of millions of documents.

Yes, it's planned!

Yes, indeed, it's crucial to have an SSD. With it, loading will be instant (a few ms).

You should definitely try Meilisearch again. We have optimized a lot of the consumption and indexation performance. Even with all the improvements, we think it's essential to continue focusing on it during 2023.

And indeed, Meilisearch uses memory-mapping, which means that everything is on disk, and it will try to take as much memory as possible. For your information, we successfully ran a 115M documents dataset on a 1Gb RAM machine.

If you remove the URLs from indexation, it'll generally save a ton of place and will be much, much faster to index. We are thinking about not indexing URLs by default; you can help us by explaining your use case here -> https://github.com/meilisearch/product/discussions/553

Just a detail, if you're making a `du -sh` on your computer, the size on the disk will stay unchanged because we are doing soft deletion ;). Don't worry. It will be physically deleted after a while if you need it in the future.

If you kept the default configuration of Meilisearch, the maximum size of the HTTP payload is 100Mb (for security). You change it here -> https://docs.meilisearch.com/learn/configuration/instance_op...

addDocumentsInBatches() is just an helper to send your big json array into multiple parts, not absolutely sure you'll need it. (Code -> https://github.com/meilisearch/meilisearch-js/blob/807a6d827...)

Hello, I'm the Meilisearch CEO, your issues could be because you sent your data without configuring your index, and it's what I read from your comment. Just change your setting to not index URLs.

Check out the doc: https://docs.meilisearch.com/reference/api/settings.html#upd... With the payload: `["title", "descriptionHTML"]`

It will change everything!

Can I get your use case? What is the data you have to index? If it's logs, indeed, Meilisearch will not be the best fit. I would recommend https://quickwit.io. If it's structured or unstructured data, even if 10TB seems huge for our use cases, I would love to put you in contact with my engineering team. We would find a way to manage it.

Hello, I would love to know what are your needs/limitations that makes you ask for cluster/sharding. Meilisearch is cloud-native, so we don't need clusters of machines to provide a highly available solution. By default, our service is highly available and vertically scalable. We don't have geo-replication today, but it will come later in 2023. I don't know your need for scalability, but we can handle almost every use case. You may only be in an Enterprise plan.

We support almost all languages. The only thing is that in some languages, without help and intervention from the community, we stop when our level of comprehension is not enough. Today we handle perfectly Latin-based languages and all languages that are space separated. We have also worked with the community to improve Chinese, Japanese, Thai, and Korean, which is under review.

Cool! Meilisearch's CEO here. We have spent so much time improving indexation speed. I'm so glad the experience is better for you now! Even more improvements will be released in the future.

Good to know. If you find some Russian language support improvement we can do, don't hesitate to create an issue on our tokeniser https://github.com/meilisearch/charabia.

Moreover, it's Hacktoberfest. If you want to help us improve the language support, it would be awesome!

Hello, I'm Quentin, Meilisearch's CEO. Meilisearch Cloud is encrypted at rest. We're using, at this time, a k8s with replicated and encrypted persistent volumes. We just didn't talk about it yet.

Thanks! Happy to get you as a user :)

Hello everyone, I'm Quentin, the CEO of Meilisearch. We are very proud to announce today our incredible fundraising. Over the last three years, we have succeeded in establishing ourselves as a serious competitor in the search industry. In just 3 years, we have managed to win over the hearts of developers by enabling them to create fast, relevant, and easy-to-implement search experiences in their applications.

But this was only the beginning. Thanks to this new investment, we will continue to invest in open source and the community and start the next big phase of our company. Offering the same quality of research to companies on a completely different scale.

We have a lot of ideas to make Meilisearch even more scalable and easier to integrate into the devs stack. Geo-replication, data sync with primary databases, search bar customizable in one line of javascript, and many more.

Stay tuned for news. I will be happy to answer all your questions here.

Indeed for the moment, I would not advise going to more than a few million documents, especially if you want to make regular updates. With the new update that we will release in March, I think we will go up to a few tens of millions of documents with much higher update frequencies. However, we keep in mind that Meilisearch should do much, much more. This will be a work in progress for us during the year, as we plan in the future to allow Meilisearch to be no longer a search binary with limited resources but as a globally distributed solution that will have no limits to scale. But it's still too early to talk about our long-term plans on how we will improve user-facing search. #serverless

To answer your third point. Indeed, each solution has its own positioning, which implies requirements and trade-offs. I don't know the positioning of Typesense and Quickwit so I would have difficulty speaking on their behalf. But I have the impression that Quickwit is the new Elasticsearch. In the long term, they will surely position themselves in front of them, put themselves on the log market, and why not of the security analysis. It is impossible to update or delete data makes me think of this. A system capable of managing terabytes of data with precise and fast tasks. Even if it is far from the need of instant search.

On our side, we have a clear focus on user-facing search. All searches can improve the user experience and increase user satisfaction and product retention. Use cases today are mainly E-commerce, marketplace, site-search, media, SaaS application, B2C applications. The needs for these use cases are the same. Search in a fixed data type with incredible relevance and performance.

Moreover, I want to underline this because I see that you mention it in your comments. At Meilisearch, we have made it a point of honor to offer a solution with the best possible developer experience. Everything we do (API, SDK, Documentation), we do it with developer experience in mind.

Hello @joshxyz, here is the CEO of Meilisearch.

I wanted to thank you for your feedback. I also wanted to confirm that several people have given us feedback on indexing speed problems.

That is why we have decided to focus on this issue during the first quarter of 2022. We have already made a lot of progress on indexing speed, and we can't wait to show you these new capabilities with the arrival of v0.26.0.

If you want another comparison table we have also made our own. https://docs.meilisearch.com/learn/what_is_meilisearch/compa...

Hello, as CEO of MeiliSearch, I'm really sorry from the whole team if we did not satisfy you when solving one of your bugs. I don't know which bug exactly you are referring to, but in any case, we try to answer our users and contributors with the maximum of transparency and love.

Moreover, we are certainly, on some features, maybe a little bit late. Delay that we will more than compensate before the end of the year. Our priority until now has been to offer a robust search engine accessible to all. For us, the developer experience is really important, whether it is in the use of the API or in the communication with the community.

We will continue to try to do our best for the community. If you want to help us to improve, I would be happy to take your feedback.

Yes, It's marked for Q3 because it's a pretty complex feature. And we had a lot of other features to do at the same time. But the good news is that it's very well advanced and is likely to be released in mid-Q2.