Hacker News Reader

Comment by skeptrune | Hacker News Reader

Really surprised that it's faster than ripgrep given its using OpenAI embeddings. Every API call is taking 300ms off the jump given that.

View on HN

This timing is for making queries after the index is created, so it’s only two API calls for embedding / re-ranking. Overall, during testing Codebased took ~1 second to search the Linux kernel and ripgrep took ~2 seconds because it’s expensive to read gigabytes of text from disk. It’s slower for small projects, but I’m working on that.

Can you share the 2 second benchmark of the Linux kernel for ripgrep? On my workstation, ripgrep can search the Linux repo (using default settings) in under 100ms.

ripgrep generally shouldn't be reading from disk. If you're benchmarking Codebased after an initial indexing step, then ripgrep should be benchmarked on a warm cache IMO.

That's definitely fair. Running the command `rg "amd_pci_dev_to_node_id"` with default settings on the Linux kernel source tree 5 times takes: 2.48s, 1.39s, 0.816s, 0.448s, 0.438s. I'm using an M2 Max Macbook Pro.

The first time is consistent with what I reported above (I only ran the command once), but the subsequent runs are definitely faster than what I had reported and the last two runs are definitely faster than Codebased. Sorry, that's my bad. With semantic search + re-ranking disabled, that query runs in 102ms on Codebased, but that's not the default.

Thanks for making ripgrep!

I get 0.371s on my M2 mac mini, which is right in the neighborhood of that, so that makes sense. Now on my i9-12900K, I get 0.096ms.

Thanks for checking!

Reply on news.ycombinator.com