ashvardanian

After publishing this a few days ago, 2 things have happened.

First, it tuned out that StringZilla scales further to over 900 GigaCUPS around 1000-byte long inputs on Nvidia H100. Moreover, the same performance is obviously accessible on lower-end hardware as the algorithm is not memory bound, no HBM is needed.

Second, I’ve finally transitioned to Xeon 6 Granite Rapids nodes with 192 physical cores and 384 threads. On those, the Ice Lake+ kernels currently yield over 3 TeraCUPS, 3x the current Hopper kernels.

The most recent numbers are already in the repo: https://github.com/ashvardanian/StringWa.rs

Largely true, but there is always a caveat.

On Zen4 and Zen4c the register is 512 bits wide. However, internally, many “datapaths” (execution units, floating-point units, vector ALUs, etc.) are 256 bits wide for much of the AVX-512 functional units…

Zen5 is supposed to be different, and again, I wrote the kernels for Zen5 last year, but still have no hardware to profile the impact of this implementation difference on practical systems :(

My expectation, they will perform great! I’m now mostly benchmarking on 192 core Intel, AMD, and Arm instances on AWS, and in some workloads they come surprisingly close to GPUs even on GPU-friendly workloads, once you get the SIMD and NUMA pinning parts right.

For BioInformatics specifically, I’ve just finished benchmarking Intel SPR 16-core UMA slices against Nvidia H100, and will try to extend them soon: https://github.com/ashvardanian/StringWa.rs

Sadly, no! On the bright side, they support new AVX2 VNNI extensions, that help with low precision integer dot products for Vector Search!

SimSIMD (inside USearch (inside ClickHouse)) already has those SIMD kernels, but I don’t yet have the hardware to benchmark :(

For Vector Search the top 2 are: Meta’s FAISS and (my) Unum’s USearch. Lucene powers Elastic, Solr, MongoDB Atlas, AWS OpenSearch, Azure Cognitive Search. USearch powers ClickHouse, DuckDB, YugaByte, TiDB, ScyllaDB, MemGraph, KuzuDB, Lantern, and a few big closed source names that don’t mention it, as far as I know. FAISS has the highest usage among Python developers, but if you are indexing large collections you should consider alternatives.

Nice observation!

Took me a while to realize that Grace Blackwell refers to a person and not an Nvidia chip :)

I’ve worked with large genomic datasets on my own dime, and the default formats show their limits quickly. With FASTA, the first step for me is usually conversion: unzip headers from sequences, store them in Arrow-like tapes for CPU/GPU processing, and persist as Parquet when needed. It’s straightforward, but surprisingly underused in bioinformatics — most pipelines stick to plain text even when modern data tooling would make things much easier :(

Lucene is tough to deal with. About 15 hours ago — right when this comment was posted — I was giving a talk at Databricks comparing the world’s most widely used search engines. I’ve never run into as many issues with any other similar tool as I did with Lucene. To be fair, it’s been around for ~26 years and has aged remarkably well... but it’s the last thing I’d choose today.

The question is how many, and what kind of VMs you use? It greatly affects performance :)

I run a lot of search-related benchmarks (https://github.com/ashvardanian) and curious if you’ve compared to other engines on the same hardware setup, tracing recall, NDCG, indexing, and query speeds.

Very curious about the hardware setup used for this benchmark!

Related, but wasn’t broadly discussed on HN: https://astral.sh/blog/wheel-variants

Blogpost about the recent license change: https://albumentations.ai/blog/2025/01-albumentationsx-dual-...

Was on the front page under 24h ago: https://news.ycombinator.com/item?id=45167401

In relation to other core specs, like transistor and core counts and memory bandwidth, 50% growth in 10 years is pretty much flat-lining, I’d say.

There's a nice write-up by Cloudflare from July covering some of those charts: https://blog.cloudflare.com/ai-search-crawl-refer-ratio-on-r...

I don’t know how I forgot to post this library and write-up on HackerNews. It’s an unusual thread-pool implementation that focuses entirely on simultaneous multithreading, skipping task queues, futures, and asynchrony. That makes workload distribution much cheaper than in Taskflow (C++) or Rayon (Rust), while getting closer to OpenMP in raw performance.

The design avoids memory allocations, focuses on lock-free and CAS-heavy atomics, and leans on modern hardware instructions for busy-waiting and NUMA-friendly execution. It still needs polishing, and I’m always open to feedback on how to push it further.

Any good comparisons of traditional embedding models against embeddings derived from autoregressive language models?

I've been struggling to find the settings to control NUMA on public cloud instances for a long time. Those are typically configured to present a single socket as a single UMA node, even on huge EPYCs. If someone has a tip on where to find those, I'd appreciate a link!

You can avoid hard-coding the whitespace symbols and have a generic byte-set search kernel via `vpshufb` AVX512BW-capable CPUs [1] or via `tbl` instructions on NEON-capable CPUs [2].

[1]: https://github.com/ashvardanian/StringZilla/blob/2f4b1386ca2...

[2]: https://github.com/ashvardanian/StringZilla/blob/2f4b1386ca2...

PS: Finding CPUs that support AVX-512 and SVE is relatively trivial - practically every cloud has them by now. It's harder to find Arm CPUs with wide physical registers, but that's another story.

I like that more people are getting involved with SIMD, and there have been several posts lately on both memmem-like and memcpy-like operations implemented in SIMD in different programming languages.

In most cases, though, these still focus on AVX/NEON instructions from over 10 years ago, rather than newer and more powerful AVX-512 variations, SVE & SVE2, or RVV.

These newer ISAs can noticeably change how one would implement a state-of-the-art substring search or copy/move operation. In my projects, such as StringZilla, I often use mask K registers (https://github.com/ashvardanian/StringZilla/blob/2f4b1386ca2...) and an input-dependent mix of temporal and non-temporal loads and stores (https://github.com/ashvardanian/StringZilla/blob/2f4b1386ca2...).

In typical cases, the difference between the suggested SIMD kernels and the state-of-the-art can be as significant as 50% in throughput. As SIMD becomes more widespread, it would be beneficial to focus more on delivering software and bundling binaries, rather than just the kernels.

Singletons are considered an antipattern in pretty much all PLs. C++, mentioned in the article, is not an exception.

First, “Accept all cookies”, and now this. The competition for the worst web UX is heating up.

$500B is a huge headline number—even by the standards of the world’s largest lossy compression competition. But I don’t think anyone remotely familiar with the space ever expected the number to be accurate, or even cared that much about the total budget. It’s mainly a signal of political will, I assume. Mega-projects always take longer than expected and come with unpredictable costs. That said, OpenAI still has an edge over the competition, and I doubt there are many team better at raising capital than theirs.

It’s true that they compare to different Intel CPUs in different parts of the webpage, and I don’t always understand the intentions behind those comparisons.

Still, if you decode the unreadable footnotes 2 & 3 in the bottom of the page - a few things stand out: avoiding AMX, using CPUs with different core-counts & costs, and even running on a different Linux kernel version, which may affect scheduling…

> All vector units have full 512 bits capabilities except for memory writes. A 512-bit vector write instruction is executed as two 256-bit writes.

That sounds like a weird design choice. Curious if this will affect memcpy-heavy workloads.

Writes aside, Zen5 is taking much longer to roll out than I thought, and some of AMD's positioning is (almost expectedly) misleading, especially around AI.

AMD's website claims Zen5 is the "Leading CPU for AI" (<https://www.amd.com/en/products/processors/server/epyc/ai.ht...>), but I strongly doubt that. First, they compare Zen5 (9965), which is still largely unavailable, to Xeon2 (8280), a 2 generations older processor. Xeon4 is abundantly available and comes with AMX, an exclusive feature to Intel. I doubt AVX-512 support with a 512-bit physical path and even twice as many cores will be enough to compete with that (if we consider just the ALU throughput rather than the overall system & memory).

That's very informative, thanks! So a DGX H200 should be able to run it at 16-bit precision. If I recall correctly, the current hourly rate should be around $25. Not sure what the throughput is, though.

Thanks for the uploads! Was reading through the Unsloth docs for Qwen3-Coder before I found the HN thread :)

What would be a reasonable throughput level to expect from running 8-bit or 16-bit versions on 8x H200 DGX systems?

There’s an extensive change-log supporting the CCCL 3.0 release on GitHub from 3 hours ago: https://github.com/NVIDIA/cccl/releases/tag/v3.0.0

In case you are searching for fun use-cases, here's how one experiment with weird similarity metrics & kNN data-structures via Cppyy (for C++ kernel), Numba (for Python), or PeachPy (for x86 Asm), interacting with a precompiled engine: https://github.com/unum-cloud/usearch/blob/main/python/READM...

The title made sense until the comma, and then it didn’t :)