Hacker News Reader

Comment by nojito | Hacker News Reader

Even shorter with r and probably a good deal quicker

r -e "data.table::fread(data.csv)[col1 != "foo", .(1,3)]"

I don't know anything about R and I couldn't get this to work. First, I had to install R. Then it told me that 'data.table' was unrecognized. So I searched around and installed that. Then, 'data.csv' is not quoted. So fixed that. So I tried running it on some data I have:

    > data.table::fread("version_downloads.csv")[date == "foo", .(1,3)]
    Error: character string is not in a standard unambiguous format
    Execution halted

Dunno how to fix this now.

Fix your strings. :)

That's not helpful. CSV data is what it is. If you're advertising an alternative for processing CSV data but it chokes on valid CSV, then that's kind of a show-stopper IMO.

And I have no idea which string to fix or how to even fix it. The failure mode here is not great.

Might be the datatypes. fread automagically tries to determine the datatypes of your columns by jumping around the file as it's being read.

https://www.rdocumentation.org/packages/data.table/versions/...

Or you can pass colClasses = 'character' as an argument to read everything as a string, but that will be much much slower.

fread also reads from stdin so you can pass bash commands into the first parameter instead of using bash to run r via R -e.

I guess that's nifty, but it makes it much less universal. It's also not clear to me why reading everything as a string would be slower. If anything, it seems to me like it would be faster, since you wouldn't need to do type guessing.

Full disclosure, I'm the author of 'xsv' and the thing I was most interested in here was performance.

Reply on news.ycombinator.com