Hacker News Reader

To start with, we can add stars (pay with prestige). Getting more into science fiction--but very possible science fiction--we can put data on the blockchain and let people transact. The data owner would get the lion's share of the transaction.

View on HN

Hmm. I can see some ways for this to work.

Awhile ago I started a project converting government voting records (both elections and congressional) into a database. Would that be interesting to you?

Here's an idea: You could also host a data bounty program, and/or start a grant program for the production of these data sets.

Still missing an answer to "what are you requirements"? How do you verify data quality, etc? What format(s)?

Currently, we support 2 "targets" a Pandas DataFrame and a file. Files can be any format. The Quilt build logic uses Pandas to read files into DataFrames so any format Pandas can read should work in Quilt to create a DataFrame node.

We support arbitrary data formats in that Quilt falls back to a raw copy if it can't parse the file. On the columnar side (things we convert to Parquet) we support XLS, CSV, TSV, and actually anything that `pandas.read_csv` can parse. We use pandas and pyarrow for column type inference. We want to add a "data linter" that checks data against user-provided rules, and welcome such feature requests on GitHub or in our Slack Channel.

Reply on news.ycombinator.com