Hacker News Reader

Comment by hambandit | Hacker News Reader

I learned some haskell as my hobby language a few years back. It was very cool and forced me to think about programming differently (and finally grok recursion). It never felt like a good language for data analysis to me though (maybe that's cause this library wasn't around? lol). This isn't meant a criticism of this library, instead, I'm curious the use cases the author, if you're around or a user, has in mind. Congrats on the v1 release!

View on HN

Author here. At the time I worked in fraud detection and we needed to automate file generation for our BRMS. Initially created this to experiment with “models as dataframe expressions” and Haskell is great for DSL-like stuff. That work is still on going: https://github.com/DataHaskell/symbolic-regression and dataframe has a native sparse oblique tree implementation.

As it’s grown it’s been pretty cool to have transparent schema transformations instead of every function mapping a statement a dataframe you can have function signatures like:

``` extract :: TypedDataFrame [Column "price" (Maybe Double), Column "quantity" Int, Column "comments" T.Text] -> TypedDataFrame [Column "price" (Maybe Double), Column "quantity" Int] -- body of extract

transform :: TypedDataFrame [Column "price" (Maybe Double), Column "quantity" Int] -> TypedDataFrame [Column "price" Double, Column "quantity" Int] -- body of transform

clean :: TypedDataFrame [Column "price" (Maybe Double), Column "quantity" Int, Column "comments" T.Text] -> TypedDataFrame [Column "price" Double, Column "quantity" Int] clean = transform . extract ```

But you can also do the simple thing too and only worry about type safety if you prefer:

``` df |> D.filterWhere (country_code .==. "JPN") |> D.select [F.name name] |> D.take 5 ```

Being able to work across that whole spectrum of type safety is pretty great.

Reply on news.ycombinator.com