Hacker News Reader

Comment by johnmyleswhite | Hacker News Reader

I am definitely going to use embedded SQLite in some of future prototypes. The only problem is that SQLite doesn't, to my knowledge, support all of the features I'd like Julia to support -- including columns that contain arrays, dictionaries and even complete tables. But, that said, it supports >90% of the features I'd be interested in.

View on HN

Ah, that's a very succinct reason. Because my needs are somewhat ordinary, I often forget about the notion that columns might be anything other than vectors containing single values. I wonder if some of the more complex datatypes could be added to SQLite in the same sort of way that Spatialite has added geospatial geometries as column types. In fact, I wonder if some of those geospatial types might be abused to push past the 90% use case barrier.

Hmm... Now I'm wondering what level of extension to SQLite would be required to create a shared library that could be plugged in to any of the various languages that need dataframe functionality. DataLite?

I'm not sure. I think, for many users, SQLite is sufficient, which is why I'll be using it for prototype work in the future. Trying to add things like arrays seems like a lot of work, since you also need to add things like HQL's EXPLODE at the same time.

Check out continuum's blaze project for python. Provides table and array abstractions for a variety of backends, including SQL solutions.

Reply on news.ycombinator.com