Check out Manticore Search for your use case. It's open-source, cost-effective, and doesn't require keeping everything in memory.
Key points:
- Columnar Storage: Efficiently handles large datasets on disk, ideal for terabyte-scale data. It's not enabled by default but can be set up easily with "CREATE TABLE ... ENGINE='columnar'".
- Faceted Search: Probably easier than anywhere else with just "FACET <field name>" added to your "SELECT" query.
- MySQL Protocol and SQL Support: If you’re familiar with SQL and MySQL, it's easier to get started compared to other search engines.