Show HN: DataHub, open source datasets for Artificial Intelligence | Hacker News Reader

Show HN: DataHub, open source datasets for Artificial Intelligence

14 pointstheo319 comments

Hi HN!

I am an undergrad student trying to build interesting things with AI.

Recently, I was looking for a dataset I could use for a new project. I realized that it is really frustrating to go through all the government websites (with terrible UX) just to find some usable dataset.

I set out to build a GitHub for datasets, named DataHub. Right now, we have more than 1000 datasets from Montréal and New York City, with more cities coming soon (and possible government agencies).

All of this is wrapped into a powerful search. It's a breeze to find a dataset to work on.

I'd be interested to know what you guys are looking at when searching for datasets and if DataHub could be of any help!

https://datahub.now.sh/

Very cool! The interface is really beautiful and I would love if data.gov was formatted like this.

What is your strategy for acquiring these datasets? Are you going to pull them from data.gov and other websites?

What happens if those datasets are changed on data.gov, will you detect that?

Hi!

For now, I only save the links to the data on other websites. This way, the data is always up to date.

In order to acquire those datasets, I wrote a couple of scrapers using chromeless.

There's a typo. "Datahub has more then 1200 datasets" should read "Datahub has more than 1200 datasets"

Thank you! I'll fix it right now

Nice work. You definitely should get in touch with the Dat Project folks. There are several of them on the core team and in the community who are actively scraping government websites for open data.

Thanks! I've never heard of them, will get in touch!

Would love to use this once there's some kind of public health / medical data of some sort!

Do you know where I could find that kind of data?

Just added US Health Data (around 2,000 datasets): https://datahub.now.sh/u/ushealthgov

I'd love to get your feedback on this!

Reply on news.ycombinator.com