Hacker News Reader

Comment by rektide | Hacker News Reader

I saw a project in similar space- Malloy- go by recently, in a post, What Happened to the Semantic Layer, which I thought nicely setup the space & problem. https://carlineng.com/?postid=semantic-layer#blog https://news.ycombinator.com/item?id=35715410

They indeed mentioned Cube.js.

Bunch of Malloy links over time: https://hn.algolia.com/?dateRange=all&query=Malloy&sort=byDa...

And cube.js, https://hn.algolia.com/?dateRange=all&query=cube.js&sort=byD...

It's all very interesting to me because I get the sense these folks feel like we haven't figured out to to make meaning from the data we have. As a developer, that rings true; the developee is the agent responsible for understanding what we have in sql now & how to eek out meaning, how to structure more info in. The information architectures are so occluded & concealed, so stoggily low level. I love these attempts to try to help us think more about our data.

View on HN

So good you brought Malloy here. I like it quite a bit because the folks really try to innovate (heck, they even have their own data querying syntax to replace SQL). But what I like even more — being part of the Cube team — that the "cons" of existing solutions that Carlin mentions in his blog are actually already solved by Cube.

With Cube, Data exploration, ideation on the data model, querying, and bringing the insights all the way down to BI tools or data apps takes minutes rather than hours or days. Done, case closed :-)

My understanding of Cube is that iterating on the data model requires the user to (1) write SQL to develop a metric (2) edit YAML or JS config to incorporate the new metric (3) issue API request to Cube server and (4) compare results to raw SQL. Am I mistaken? Does Cube offer a smoother way to do this exploration/iteration?

Hey, Carlin! Nice to see you here in comments! (Waving "hi" to the Malloy team.)

Usually, the experience would look like this: one directly develops the data model in YAML (with only bits of SQL, if needed) and instantly explores metrics. No need to start with SQL in a separate tool/place (1), no need to use the API to check metrics (2) (for that, we have Playground, an interactive UI tool), and, thus, no need to compare results to raw SQL (4). You iterate but changing the data model and seeing the metrics in an instant, quite similar to how you work with Malloy, if I may.

My operating theory is that in app centric development the data store is a component of the application. As a result, much of the metadata that changes data into information (data with meaning) is stored in app code and config. Upon ETL to reporting, warehouse, lake, or the like, the app’s semantic layer is lost.

It's a crying shame how little of the data-core makes it's way out of applications. For both the app's owning entity & especially for the users.

I paid respects[1] recently yet again to Window Manager Improved Improved (wmii), which kept it's state in a 9p filesystem any user could easily browse & modify. The state is still a component of the app/window manager, but it's at least malleable to all.

[1] https://news.ycombinator.com/item?id=35768686

Reply on news.ycombinator.com