Hacker News Reader

Comment by adolph | Hacker News Reader

They seem to be doing lots of work but I don’t understand what customer value this creates.

What does a backing data lake afford a Notion user that can’t be done in a similar product, like Obsidian?

The whole point of a data warehouse is that you can rapidly query a huge amount of data with ad hoc queries.

When your data is in Postgres, running an arbitrary query might take hours or days (or longer). Postgres does very poorly for queries that read huge amounts of data when there's no preexisting index (and you're not going to be building one-off indexes for ad hoc queries—that defeats the point). A data warehouse is slower for basic queries but substantially faster for queries that run against terabytes or petabytes of data.

I can imagine some use cases at Notion:

- You want to know the most popular syntax highlighting languages

- You're searching for data corruption, where blocks form a cycle

- You're looking for users who are committing fraud or abuse (like using bots in violation of your tos)

From the article: "Unlock AI, Search, and other product use cases that require denormalized data"

1st paragraph: "Managing this rapid growth while meeting the ever-increasing data demands of critical product and analytics use cases, especially our recent Notion AI features, meant building and scaling Notion’s data lake."

Beyond the features that the sibling comment mentioned, this kind of data isn’t really for end users. It’s a way that you can package it up, “anonymize” it, and sell the data to interested parties.

Is that why they’re putting images in Postgres? I don’t understand that design decision yet.

I... Don't think they are? If you look at the URL for images in notion, you can see the S3 hostname.

Notion employee here. We don't put images themselves in Postgres- we use s3 to store them. The article is referring to image blocks, which are effectively pointers to the image.

For someone like Notion, they probably aren't selling this data. The primary use case is internally for analysis (eg product usage, business analysis, etc).

It can also be used to train AI models, of course.

That "probably" is doing a lot of heavy lifting. That said, whether they sell it or not, it's all that data that is their primary value store at the moment. They will either go public or sell, eventually. If they go public, it'll likely be similar to Dropbox; a single fairly successful product, but failing attempts to diversify.

"Selling" is a load-bearing word, too. They're probably not literally selling SQL dumps for hard cash. But there are many ways of indirectly selling data, that are almost equivalent to trading database dumps, but indirect enough that the company can say they're not selling data, and be technically correct.

Reply on news.ycombinator.com