Hacker News Reader

Comment by lxgr | Hacker News Reader

The example is failing at decompression for me, and I suspect it's because of this:

> That data comes as a zipfile, which gunzip will decompress.

Doesn't gunzip expect gzip files, as opposed to ZIP (i.e. .zip) files?

On Linux I get further (apparently Linux gunzip is more tolerant of the format error than the macOS default one), but there, I then run into:

> Error: no such column: Date

View on HN

Here's the gzipped tarball of the docs you can read to find out

https://www.gnu.org/software/gzip/manual/gzip.txt.gz

This is slightly abusive of gunzip, but it does work, but only because the zipfile in question only contains a single file and you are piping it in via shell. If you had tried to download the file and then run gunzip on it the program would have refused.

As for the second error, I think you might be trying to import an empty file or maybe an error message? Probably related to the first problem.

> but it does work

Only on some gzip implementations, e.g. the one shipping with many Linux distributions. It doesn't work on macOS, for example.

the second error is because you need to add a `-csv` argument to sqlite

GNU gzip has limited support for ZIP:

> Files created by zip can be uncompressed by gzip only if they have a single member compressed with the “deflation” method. This feature is only intended to help conversion of tar.zip files to the tar.gz format.

-- https://www.gnu.org/software/gzip/manual/html_node/Overview....

The example works on my MacBook if I replace gunzip with funzip. Also, I like to see the value. So:

    curl -s https://www.ecb.europa.eu/stats/eurofxref/eurofxref-hist.zip | funzip | sqlite3 -csv ':memory:' '.import /dev/stdin stdin' "select Date,USD from stdin order by USD asc limit 1;"

Thx. I have extended a bit as a script:

    # !/bin/bash
    ARG1=${1:-'select Date from stdin order by USD asc limit 1;'}
    curl -o - -Ls https://www.ecb.europa.eu/stats/eurofxref/eurofxref-hist.zip \
  | funzip | sqlite3 -csv ':memory:' '.import /dev/stdin stdin' "${ARG1}"

    # Usage:
    $ ./run.sh 'select * from stdin limit 1;'

At least in Ubuntu it doesn't do anything at all.

# gunzip -d master.zip gunzip: master.zip: unknown suffix -- ignored

It's a ZIP file, so you'll want to use unzip, not gunzip.

Ubuntu's version of g(un)zip actually does tolerate ZIP inputs though; for that usage, just omit the '-d'.

Per the docs (I linked them in another comment), the correct command is `gunzip <foo.zip` or `gunzip -S .zip foo.zip`.

Reply on news.ycombinator.com