LucidDB | Hacker News Reader

LucidDB

luciddb.org

23 pointsparaschopra9 comments

"LucidDB is the first and only open-source RDBMS purpose-built entirely for data warehousing and business intelligence. It is based on architectural cornerstones such as column-store, bitmap indexing, hash join/aggregation, and page-level multiversioning. Most database systems (both proprietary and open-source) start life with a focus on transaction processing capabilities, then get analytical capabilities bolted on as an afterthought (if at all). By contrast, every component of LucidDB was designed with the requirements of flexible, high-performance data integration and sophisticated query processing in mind. Moreover, comprehensiveness within the focused scope of its architecture means simplicity for the user: no DBA required."

Unintentional hilarity is hilarious.

What's hilarious about this?

It is based on architectural cornerstones such as column-store, bitmap indexing, hash join/aggregation, and page-level multiversioning

Meanwhile... no DBA required. Riiiiiiiiiiiiiiiiight.

LucidDB is the first and only open-source RDBMS purpose-built entirely for data warehousing and business intelligence.

Does LucidDB predate MonetDB? Or is MonetDB considered multipurpose? It has an XML front end in addition to SQL, but the data store is still column-oriented.

Not sure about the other features, like multiversioning, though that's an old technique (InterBase pioneered it) so I wouldn't be surprised either way.

Well, since I wrote that blurb on the luciddb website, and this ycombinator article has given a nice bounce in website hits, I guess I should respond. :)

So evdawg, I'm curious: what is it about those features that makes you think they require a DBA? Since I was chief architect at LucidEra through the end of last year, I can tell you that we ran many many LucidDB instances with no DBA at all, just well-designed applications (of high complexity) and automatic management frameworks. The netops team never had to mess with LucidDB; they just kept the servers running.

From the MonetDB website: "MonetDB is a open-source database system for high-performance applications in data mining, OLAP, GIS, XML Query, text and multimedia retrieval." I'd call that multipurpose.

Anyway, one day I should come up with a less inflated blurb, but hey, I'd rather be working on the code.

JVS

Is there a side-by-side technical comparison of LucidDB vs the other column stores, like MonetDB, Infobright, Vertica and Sybase IQ? I assume there are some design and implementation differences, it's just not clear to me what they are.

MonetDB: designed for super-fast in-memory execution via vector operators; tends to thrash once you go beyond what can fit in memory since the algorithms aren't designed for that. X100 is supposed to address that but no one has seen it yet.

LucidDB: optimized for the case where your data does not fit in memory. Supports a lot of additional ETL-related stuff like upsert, Java row-level transformations, foreign data extraction, warehouse labels, hot backup.

Infobright: don't really know; their community edition is intentionally limited (e.g. doesn't support DML). Slightly smoother transition if you are already a MySQL shop.

Vertica: clustered, optimized for taking advantage of multi-sorted materializations, pricey (OK, the last one is non-technical).

Sybase IQ: don't know.

Thanks! I like the ETL facility. Does LucidDB need any help?

Always...

Reply on news.ycombinator.com