HermitX

I hope you and your sharp opinions make you happy. You don't need to worry about what kind of person I am.

I believe that everyone's thoughts are influenced by the information they receive. We have some stereotypes about certain countries and peoples. But in reality, we have neither been there in person nor do we know any of the people from there. Our impressions of them are entirely imposed on us by the media, and it's truly sad that we have a negative view of them because of these impressions.

Thank you for your great job!

As long as you own it, you will love it. Trust me.

Thanks for sharing, I really like Postgres. However, I generally use Postgres for OLTP work. I would like to point out two things:

1. Based on Clickbench results, pg_analytics is still far from top-tier performance. If you're looking for a high-performance OLAP database, you should consider the top-ranked products.

2. The queries tested by Clickbench are really simple, far from real data analysis scenarios. You must be aware of the TPC-DS and TPC-H benchmarks, because ClickHouse simply cannot run these test suites, so their Clickbench does not include these datasets.

Lastly, I want to say, if your enterprise is of a certain size, separating OLTP and OLAP into two databases is the right choice, because you will have two different teams responsible for two different tasks. By the way, I use StarRocks for OLAP work.

I really like Postgres, but I still don't think Postgres can do everything. From my own experience, when the scale of data becomes very large, when you need to analyze massive amounts of data, your Postgres always seems somewhat inadequate. I think Postgres performs perfectly when dealing with OLTP-type workloads. However, if you need more OLAP support, I recommend you use StarRocks (www.starrocks.io). We now import data from Postgres into StarRocks for data analysis, and the experience is fantastic. Moreover, StarRocks also supports direct queries on data lakes, which is incredible.

The work on materialized views in ClickHouse, from my perspective, is still quite rudimentary. If interested, I suggest looking into the materialized views of StarRocks. StarRocks' materialized views not only support multi-table joins and incremental updates but can also automatically rewrite query requests, so you don't have to modify your SQL statements at all.

https://celerdata.com/glossary/materialized-view

Whether you acknowledge it or not, in the journey of human progress, some individuals have played roles far greater than others. If our goal is to advance the progress of humanity as a whole, then rewarding these individuals is usually justified. Personally, I believe that Musk is greater than Buffett.

I think you're talking about doing denormalization before importing data into an OLAP system to avoid subsequent joins. However, this greatly limits the flexibility of data modeling. Moreover, denormalization can be a headache-inducing process. In fact, I have tested StarRocks (https://github.com/StarRocks/starrocks), and it is capable of performing joins while streaming data imports, and the speed is very fast. It's worth giving it a try.

Excellent learning material, thanks for sharing. I've noticed an interesting trend: JOINS are crucial for data analytics, yet many new open-source data analytics products, or open-source OLAP products, offer limited support for JOINS. Examples include ClickHouse, Apache Druid, and Apache Pinot. It seems that currently, only Trino and StarRocks provide robust support for JOINS. Commercial products tend to have better support in this area. I hope the various open-source projects will also enhance their JOIN capabilities.

Comparison of the Open Source Query Engines: Trino and StarRocks

In fact, your choice is determined by your business scenario. I believe that overall, among these databases, StarRocks offers the highest flexibility. Particularly, if your business scenario relies on performing data analysis through joins across multiple tables, then StarRocks is almost your only option. Additionally, if you want to work with ANSI SQL, then you might have to use StarRocks.

Shopee is using StarRocks to build its next-generation data analytics system.

Tencent Games is using StarRocks to build its next-generation analytics system.

Guys, I think StarRocks is truly a promising open-source analytical database.

Real-time Analytics at Airbnb

StarRocks: The #1 Solution on ClickBench

StarRocks seems to be at the top position of Clickbench. https://benchmark.clickhouse.com/

How To Reduce Snowflake Costs by 80%

UNDERSTANDING CLICKHOUSE: BENEFITS AND LIMITATIONS

An introduction to StarRocks project

How vectorization improves database performance

Real-time analytics is not easy.