Hacker News Reader

Comment by snidane | Hacker News Reader

Array notation is great, but only for single array operations and for dense array operations.

The moment you need to run complex group bys and store non-contiguous data efficiently, it gets awkward pretty quick.

On the other hand, operations on dense data is pretty cumbersome in SQL. You can only do so much with limited support of proper scan algorithms, merge joins or bolted on window functions.

Please somebody combine APL with SQL and you win the programming language wars.

View on HN

> Please somebody combine APL with SQL and you win the programming language wars.

kdb+ and q fit this description. Docs here: https://code.kx.com/q/basics/qsql/

Here's an example.

In SQL:

    SELECT stock, SUM(amount) AS total FROM trade GROUP BY stock

In q:

    q)select total:sum amt by stock from trade

Was gonna say the same plus kdb+ is a columnar store so you can get vectorizarion in your sql execution as well.

I would say APL-family languages today have largely addressed these concerns with operators such as Key[0][1] and nested arrays. J also has built-in support for sparse arrays. Some more complicated things like storing a list of lists in a compact representation (perhaps lengths and data) aren't supported natively, but I'd consider that a niche concern as a list of lists will have similar performance, just with more memory use.

There's a lot of database software built on array languages, with kdb and Jd being the most prominent as far as I know.

[0] https://aplwiki.com/wiki/Key

[1] https://code.jsoftware.com/wiki/Vocabulary/slashdot#dyadic

R? Julia? Python with Pandas even.

Reply on news.ycombinator.com