Hacker News Reader

zX41ZdbWon Show HN: misa77 - a codec that decodes 2x faster than LZ4 (at better ratios)

Just tested a few minutes ago on the ClickBench dataset. Overall quite good, but, depending on particular columns, most of the time slower - e.g., ~2.0 vs ~2.8 GB/sec on Graviton 4 machine in AWS.

zX41ZdbWon Show HN: misa77 - a codec that decodes 2x faster than LZ4 (at better ratios)

You can look at the LZ4 decompression implementation in ClickHouse: https://presentations.clickhouse.com/2018-highload-siberia/ (the presentation is quite old, but the implementation was updated recently). It uses the same LZ4 format, just decompresses faster.

zX41ZdbWon Show HN: misa77 - a codec that decodes 2x faster than LZ4 (at better ratios)

It is slower than LZ4 on AArch64.

zX41ZdbWon Helicone allows users to write SQL directly to a shared ClickHouse

Great article! About inserting LIMIT - we have out-of-band `limit` and `offset` settings:

https://play.clickhouse.com/docs?user=play#q=limit&name=limi...

zX41ZdbWon Pg_re2: Postgres extension for fast, RE2-powered regular expressions in Postgres

ClickHouse also has vectorscan as a regexp engine, supporting multi- and fuzzy-matching, multi-Volnitsky for substring matching, as well as the new LLVM-JIT compiled regexp engine.

zX41ZdbWon l: A new runtime for k and q

I've recently tested BQN on ClickBench (a benchmark for OLAP databases), and the results are not great: https://benchmark.clickhouse.com/#system=+N|liH&type=-&machi...

If anyone is curious how queries in this language look, you can see it here: https://github.com/ClickHouse/ClickBench/pull/939/changes#di...

zX41ZdbWon Ray Tracer in SQL

Coding with AI is not much different from coding without AI, just faster and with slightly less typing. But you still have to organize the work and split it into small items. If you ask a model to implement a ray tracer in SQL, it can do something one-shot, but it won't be as satisfactory as with an engineer.

zX41ZdbWon Ray Tracer in SQL

It is needed to find bottlenecks in the engine and optimize customer queries. For example, it gives things like: https://github.com/ClickHouse/ClickHouse/pull/108757

You can read more about this approach in my presentation: https://presentations.clickhouse.com/2026-openhouse-sf/great... and video: https://www.youtube.com/watch?v=FmS7VopaqNg

zX41ZdbWon Ray Tracer in SQL

It is needed to find bottlenecks in the engine and optimize customer queries. For example, it gives things like: https://github.com/ClickHouse/ClickHouse/pull/108757

You can read more about this approach in my presentation: https://presentations.clickhouse.com/2026-openhouse-sf/great... and video: https://www.youtube.com/watch?v=FmS7VopaqNg

zX41ZdbWon Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

I host a publicly open database with Hacker News data at https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPT...

So you can create any sort of similar services in a single SQL query and an HTML page.

I also hosted it as a publicly accessible data lake, which you can query from everywhere: https://github.com/ClickHouse/ClickHouse/issues/29693#issuec...

It is also updated in real-time.

zX41ZdbWon Ten years of ClickHouse in open source

Found one example that survived, and it was legitimate: https://github.com/ClickHouse/ClickHouse/pull/99522/changes

zX41ZdbWon Ten years of ClickHouse in open source

The CI only runs after one of the maintainers adds a "can be tested" label. When the label is already added, it stops when any of the infrastructure-related files are modified, like Dockerfile, CI configurations, etc. This is quite ok, but not 100% bulletproof, as you can easily do weird things by modifying the code or using a bug in the compiler. However, the CI infrastructure runs on isolated machines inside an isolated account.

Before adding the label, we have to check the diff for suspicious things. There were a few abuse attempts (all of them were from now vanished GitHub accounts).

zX41ZdbWon A SQL-Inspired Query Language Designed for Event Sourcing (2025)

There is already a database named EventQL: https://github.com/eventql/eventql I covered it in my presentation: https://presentations.clickhouse.com/2018-unknown_developers...

zX41ZdbWon We are retiring our bug bounty program

I've been trying for more than a year to make TursoDB load a single file, and it fails: https://github.com/ClickHouse/ClickBench/issues/336

zX41ZdbWon Comprehensive C++ Hashmap Benchmarks (2022)

The performance of hash tables and hash functions significantly depends on the data distribution, and should be compared on real datasets.

I've covered it in my presentation: https://presentations.clickhouse.com/2017-hash_tables/

zX41ZdbWon Two pilots dead after plane and ground vehicle collide at LaGuardia

Ground vehicles with transponders: https://adsb.exposed/?dataset=Planes&zoom=7&lat=42.1262&lng=...

zX41ZdbWon What CI looks like at a 100-person team (PostHog)

https://github.com/ClickHouse/ClickHouse/pull/99828

zX41ZdbWon What CI looks like at a 100-person team (PostHog)

Two problematic statements in this article:

1. Test pass rate is 99.98% is not good - the only acceptable rate is 100%.

2. Tests should not be quarantined or disabled. Every flaky test deserves attention.

zX41ZdbWon Programming in K

Similarly, this is how it was introduced in ClickHouse in 2019: https://github.com/ClickHouse/ClickHouse/pull/4774

zX41ZdbWon Information Is Beautiful

I once tried to post an interesting visualization to r/dataisbeautiful and received hundreds of upvotes, but then it was wiped for an unknown reason. Then I contacted the mods, both on Reddit and over email, to no avail.

It was a very frustrating experience, don't recommend anyone to try.

zX41ZdbWon Command-line Tools can be 235x Faster than your Hadoop Cluster (2014)

Though ClickHouse is not limited to a single machine or local data processing. It's a full-featured distributed database.

zX41ZdbWon Command-line Tools can be 235x Faster than your Hadoop Cluster (2014)

When a single server is not enough, you deploy ClickHouse on a cluster, up to thousands of machines, e.g., https://clickhouse.com/blog/how-clickhouse-powers-ahrefs-the...

zX41ZdbWon Exasol Personal – Democratizing Big Data Analytics

This is bad news: it is not usable:

> 5.3 Licensee may not disclose any benchmarking or results of evaluating the Software without Exasol´s prior written consent

zX41ZdbWon Exasol Personal – Democratizing Big Data Analytics

This is good news.

I was trying to add Exasol to ClickBench (https://github.com/ClickHouse/ClickBench/) since 2016, but it was not possible due to the limitations and the fact that it required using a custom virtual machine image.

Now we should try it again...

zX41ZdbWon The URL shortener that makes your links look as suspicious as possible

I would also like to have something like this, but for "vintage" links - something that looks like it was from the late 90s.

I use them in tests, just for fun: https://github.com/ClickHouse/ClickHouse/blob/master/tests/q...

zX41ZdbWon Sirius DB

GPU databases can run a small subset of production workloads in a narrow combination of conditions.

There are plenty of GPU databases out there: mapD/OmniSci/HeavyDB, AresDB, BlazingSQL, Kinetika, BrytlytDB, SQReam, Alenka, ... Some of them are very niche, and the others are not even usable.

zX41ZdbWon Show HN: 22 GB of Hacker News in SQLite

You can run it locally, but it is a client-server architecture, which means that something has to run behind the browser.

zX41ZdbWon Show HN: 22 GB of Hacker News in SQLite

The query tab looks quite complex with all these content shards: https://hackerbook.dosaygo.com/?view=query

I have a much simpler database: https://play.clickhouse.com/play?user=play#U0VMRUNUIHRpbWUsI...

zX41ZdbWon High-performance C++ hash table using grouped SIMD metadata scanning

The test does not look realistic: https://github.com/Cranot/grouped-simd-hashtable/blob/master...

Better to use a few distributions of keys from production-like datasets, e.g., from ClickBench. Most of them will be Zipfian and also have different temporal locality.