Hacker News Reader

Comment by moonchrome | Hacker News Reader

I'd say Postgres + pgvector is even simpler if you're doing small scale document search (eg. internal knowledge bases, documentation sources, codebase indexing, etc.).

pgvector is even supported out of the box on Azure and AWS RDS.

Just spin up a docker container [1], add a vector column to your table and you're ready for embedding search.

[1] https://hub.docker.com/r/ankane/pgvector

If you're starting out with a prototype - do yourself a favour, steer clear of the chromadb examples with langchain. In fact steer clear of langchain in general :) Just go for OpenAI API and PostgreSQL+PGVector - you'll have to do some boilerplate - but the stuff in langchain is just terrible, you'll have to rewrite it and do the boilerplate at some point anyway and this stack is super simple to deploy.

View on HN

Now that opensearch has ivfpq, and I assume pgvector will get it within 6mo... the current typical case for most teams, big or small, should be handled. Regular databases are all adding it, further chiseling away at the need for dedicated vector DB startups. This seems consistent with what we guessed would happen at the bottom of https://gradientflow.com/the-vector-database-index/ .

AFAICT standalone optimized vector stores will still have their place, like super latency sensitive or high-throughput scenarios. Unfortunately for many stakeholders, venture-scale returns to justify the megarounds from the peak vc fomo of the last few years seems unclear. The TBD hail mary here may be generative AI: even if most modern vector search workloads are largely fine with regular DB extensions to support a few kinds of vector indexes, the continued growth of generative AI, knowledge graphs, etc., may somehow grow the pie for non-standard DBs here. That's not obvious to me. For example, with Databricks LakehouseIQ, a lot of the use case may be eaten by the data warehouses.

There's also pgvector (trivial) examples for different languages, see e.g. github.com/pgvector/pgvector-python

"Just spin up a docker container" is a self-contradicting sentence. For non-complex applications, anything needing to touch containers is itself too complicated.

Even with pgvector, there's no good way to write a simple tutorial for embedding newbies.

Anything talking about Lucene is non trivial from start.

I'm coming at this from a perspective of a competent dev trying to build a tool with OpenAI APIs - which, from what I can tell, is a growing topic.

If you're experienced with building APIs and new to the LLM stuff - skip the langchain and chromadb nonsense. Just use the OpenAI APIs and pgvector.

Chromadb and langchain are usefull when writing notebook prototypes to get an idea of how this stuff works - but discard immediately after that phase and save yourself the trouble of porting later.

I recently spun up a MySQL instance outside a container for what I hope is the last time.

MySQL is very attached to /etc/mysql, /var/log/mysql and /var/lib/mysql, which makes sense if one thinks of it as a piece of a distribution and makes no sense if one thinks of it as a service that stores data in a filesystem or directory that one sets up for the purpose. Apparmor and (don’t get me started) SELinux dig this in deeper. What if you want two MySQLs on one host? What if you don’t want to mount something on /var/lib/mysql?

If mysqld were invoked by pointing it at a configuration and data and it just worked, I’d be more okay with it.

The fact that I really don’t want to couple upgrades of MySQL to distro upgrades is just icing on the cake.

> I'd say Postgres + pgvector is even simpler

For development, perhaps. For production, absolutely not.

I wish HN had some bot that would just delete any comment from people recommending installing databases. Because 99.9% of the time it's from those who have no experience in running one in a Production environment. Keeping it secure and ensuring backup/restore cycle works is seriously non-trivial.

But for an internal knowledge base backup/restore may be irrelevant (as documents are all copies of data and the database reconstructed fast at will) and well security is really not that difficult with a single system user.

What's wrong with Postgres + pgvector in production? pgvector is supported by RDS. Keeping an RDS instance running in production isn't exactly rocket science.

It's no match for lucene (in practice ES or Solr) in terms of performance and features, as it's very different model of indexing and operating. Keeping it running while 1000s of customers run search queries AND the app uses db AND users want faceting or other features is not an option.

Keeping postgres alive is considerably less work than keeping anything Lucene based alive.

Please clarify.

Would love to know how running a database is considerably less work than adding a library to your existing app.

Lucene uses a somewhat unreliable storage back end and needs to be rebuilt regularly.

Security and backups are important regardless of the technology you’re using.

Managed Postgres is a thing, probably much more common than managed lucerne. I can name three managed Postgres providers off the top of my head (AWS, Vultr, Supabase).

There is no such thing as managed Lucene. It's a library.

There are search engines e.g. Solr, ElasticSearch but those aren't what this paper is proposing.

Right. So now my existing stack needs a backup strategy that’s independent of the database. That’s really tough in production; Lucerne is adding complexity.

To be clear I object to this:

> I wish HN had some bot that would just delete any comment from people recommending installing databases.

Most applications already use a database. Pooh-poohing Postgres as a solution is silly. For many people, it’s actually going to be less complex to take to production than installing a library, especially one with its own complex storage needs.

Also: technically, isn’t Lucerne a database?

Isn't this solved by managed databases? Is there a problem with paying more and having AWS/Azure/GCP take care of the difficult parts for you?

Enabling pgvector in Azure was trivial.

I think if you're past pgvector performance you won't be listening to a random guy talking about pgvector but have a good understanding of the space.

If you're new (like I was a few months ago) save yourself the time I wasted on the noobtraps I mentioned. It scales way better than the OpenAI API for my use cases.

Lucene (or rather elastic/open search) is way overkill for my needs

I wish HN had some bot that would just delete any comment from people assuming everyone runs databases exposed to the internet.

What about using chromadb without langchain?

I am currently using it this way and it is really easy to just get started.

I don't know how well it compares with others, however.

Reply on news.ycombinator.com