Show HN: NNext.net – A Firebase-like managed vector storage for ML applications | Hacker News Reader

Show HN: NNext.net – A Firebase-like managed vector storage for ML applications

37 pointswakahiu23 comments

Hi HN. Peter here. As a machine learning engineer, I mostly think in terms of feature vectors, embeddings, and matrices. One of the most useful byproducts of deep neural networks is embeddings because they allow us to represent high-dimensional data in terms of lower-dimensional latent vectors. These feature vectors can be used for downstream applications like similarly search, recommendation systems and near duplicate detection.

As an ML engineer, I was frustrated by the lack of a datastore in which vectors are first-class citizens. As a result, most ML engineers, including myself, end up using awkward workarounds to store vectors such as arrays in SQL/NoSQL databases, stringifying vectors and storing them as text in in-memory-based caching systems such as Redis ETC. Furthermore, these systems don't allow for vector-based query operations such as nearest neighbor search. Consequently, engineers have to deploy additional approximate nearest neighbor search systems such as Facebook's FAISS or Spotify's ANNOY. These systems, while nifty and fast, are difficult to install and are costly to maintain. To address these issues, I built NNext, a managed vector datastore in which vectors are first class citizens. NNext, allows you to store vectors along with any json-blob metadata. Furthermore, NNext comes with a fast approximate nearest-neighbor (ANN) search capability.

I would love to get feedback on your experience as Data Scientist or ML engineer storing feature vectors and ANN systems. Please shoot me an email at [redacted].

I can't wait to see the vector DBs we're going to have 5 years from now when everyone needs to serve their embeddings. It's clearly early but frothy right now.

Also check out this similar co I ran into: https://www.pinecone.io/ (the CEO, EL, has some classic sketches and feature hashing papers).

This is really cool. Thanks for the link.

After reading this comment I made an image search engine for OpenGameArt using pinecone.io today: https://emh.lart.no/ogasearch/?imsi=dungeon%20crawl%20monste...

Every time I've needed something like this (3 times in my career, though none in the past 5 years) I've ended up having to implement it from scratch.

Depending on the topology of the data, different metrics are used and you can make different performance tradeoffs. And differing business use cases make it hard to find a one-size-fits-all. As I'm sure you're aware, there are dozens of fast ANN approaches, and the code implementing the ANN often is the tightest loop, making hard to be pluggable and performant.

TileDB is also quite interesting.

Best of luck though, I agree with you there's a "missing product" to be invented, I'm not sure this is exactly it, but I don't think I know any better.

I love this idea, but there's no way a managed service like this is going to work, for two reasons: Performance and compliance.

Compliance is pretty obvious and even if you don't store the feature dictionaries and only the vectors, that's a hard conversation to have with the compliance team. I GET IT, without the feature dictionaries the vectors are useless, I KNOW this is how it works from a technical point of view, but the compliance team still won't sign off on it. That's just the way of the world.

And much much more important is performance, uncompressed high dimensional features are huge and even if you use run-length-encoding or sparse vector storage in the protocol plus some lossless compression, I have trouble keeping the GPU fed from disk, let alone over the network, it's going to be multiple orders of magnitude too slow. If the benefits is not claimed to be streaming but are fast vector similarity, keep in mind I can do cosine similarity on literally millions of vectors a second on a single CPU core using vanilla numpy, this was fast enough for me to implement realtime face recognition vector search for Dubai airport, so pretty high scale operations.

I've love a self hosted version of this, optimised for I/O throughput to the GPU. That would be great.

You bring up two viable points of view.

Compliance is hard to get around. There are products that have done this by sheer force of their cyber security budgets such as large cloud providers which are HIPAA compliant. As a result, companies that require thorough compliance measures are simply not our target audience.

Your second point about performance is critical to almost all of our target audience. First of all, there are self-hosted versions of this such as Facebook’s FAISS (has GPU acceleration), Spotify’s ANNOY, Elastic search KNN search and Milvus. We plan to optimize performance using the latest ANN algorithms, caching, being as close to the wire (Non-REST) transmission of vectors, dimensionally reduction e.t.c. At the end of the day, our stack will probably not be as fast as a self-hosted version but it’ll confer other advantages that we hope ML engineers will find useful similarly to AWS RDS or DynamoDB.

Not on topic for the actual product... but please remove that terrible scrolling behaviour. I hate when websites do this, it makes the user experience (imo) much worse to just "jump" sections if I try to scroll a tiny bit.

Strongly agree! I searched for "scroll" in this thread to see if anyone else is complaining, I hate it when websites overwrite the scroll behavior, it's like I lost my control over navigating the website.

It ends up becoming an instant droplist for me. If I can't navigate how I want, then I won't visit the site until it's resolved.

What I've done for most of my research projects is just pickle a PyTorch dataset object that contains all my embeddings. The pkl file can then just be uploaded anywhere and becomes plug and play with any Torch model.

What advantages would this bring for a user like me? I guess it might make more sense for people working closer to production?

Exactly, once you have latency-sensitive applications that read-write embeddings, then you have to rethink your vector storage system. Furthermore, when your dataset grows to millions of datapoints, having a system that scales according avoids downtime and costly query lookup operations.

How would you handle an online ML application where the set of embeddings is changing such as an image recognition app where images are constantly being added and need to be deduplicated?

How's this different from Milvus?

Main thing is that NNext is fully managed - you don’t have to worry about provisioning servers, version upgrades and package installation/ dependencies. One of the main thing I’ve observed about ML engineers is that they typically don’t want to be encumbered by general software engineering / dev-ops tasks like platform management. They want to focus on data.

> provisioning servers, version upgrades and package installation/ dependencies

If that is nnext's only or main value prop, it will have to be changed.

Entire companies are built upon providing fast, managed services - Algolia, Firebase and Heroku to name but a few.

Is your point that the market is simply too small/doesn’t exist or that the value prop is simply too weak / defensible?

Not arguing over managed services, although, they are nearly a yaml file away from being made obsolete.

If there is an OSS system that does everything you do, that isn't a big differentiator. As soon as you have any sort of traction, then the second competitor will swoop in, Lyft, Gitlab, Indiegogo.

You need to have defensible differentiation. Otherwise why not just run the OSS system on K8S and sell the service?

My 2c opinion: this might have been true pre-k8s. With k8s, it's become a cinch for anyone to run a scalable system in the cloud. I think you need something more differentiated.

Minor sticking point, I want to object to the prefix "with k8s, it's become a cinch...". In my experience, this prefix is only true when followed by phrases such as "shoot yourself in the foot", "over complicate your infrastructure", &c. ;)

Recently learnt about FAISS & Annoy by Spotify, any comparisons done?

Erik Bernhardsson who was one of original author on Annoy has a pretty developed test suite https://github.com/erikbern/ann-benchmarks.

Why not use an opensource vector search engine/database like Milvus? https://github.com/milvus-io/milvus It's probably one of the most popular solution now.

Milvus is really easy to use, and it's oss.

https://github.com/milvus-io/milvus

Reply on news.ycombinator.com