VectorDB bench now support S3Vector

GitHub PR shows Vespa on the image, but I can't find Vespa results on the VDBBench website https://zilliz.com/vdbbench-leaderboard

Am I missing anything? (I love Vespa.ai)

Please note that the Redis supported there is not "Vector Sets" (the new Redis data type) but one of the indexes types of RedisSearch.

And, about such benchmarks: I tested another vector db benchmark, investigated it a bit, found that it was mostly measuring client implementation latencies and other internal inefficiencies...

In Redis with VSIM I can easily get 50k vSIM/seconds with 300 components vectors with redis-benchmark, yet when I tried to write a quick test for one of those engines I got a lot lower numbers because simply vectors are large (makes serialization in Python slow if not well coded), often these tests are written in high level languages, don't account for differences in client libraries speeds.

TLDR? Benchmarking is hard, for vector systems it is harder, and the results of most of such tests are totally irrelevant.

For context for those like myself who weren't familiar with Amazon's S3 Vectors, it's a relatively new S3 bucket type that's optimized for storing vectors for RAG and similar purposes, claiming to the be up to 90% cheaper than storing them in a regular bucket.

https://aws.amazon.com/s3/features/vectors/