Hacker News Reader

Comment by negrit | Hacker News Reader

Hello everyone. I quickly built this little tool based on public records provided by the government. I did the same a couple weeks ago with H-1Bs and I just added over 1.1million green cards records.

The data provided is not perfect so i'm still working on cleaning it but it should give you an idea of what is going on.

If you have any questions, feel free to ask.

View on HN

This is awesome Theo. When can I have data about Americans in France?

I can't seem to find the H1B statistics page, maybe I'm just looking in the wrong place though. Got a link?

"If you have any questions, feel free to ask."

What tools did you use/craft to clean the source data?

PostgreSQL :)

so just dump the data in and process via sql. What about pre-processing ie: different data sources (pdf, cvs), incomplete or overlapping data? I imagine some code had to be written to do this?

All my datasource where csv and mdb files. So, I directly imported them to PostgreSQL. Very little code was written to clean the data. 99% of the cleaning was done with SQL queries.

The code is mostly used to display the data. Cleaning the data with SQL queries is much faster than writing code.

Great tool! There seems to be a bug though when I drill down to year plus country of citizenship and then try to filter by job description and state or city. The job description search term ends up in the state input field.

Also, filter by company would be great!

My bad.

Fixing it ASAP

This looks great! Can the green card data be shown by categories? Like how many green cards applications were filed in each category like EB1, EB2, etc.

Neat tool, though it's a little disconcerting to be able to easily find my application - not your fault, though, the data was already there!

kinda weird that all the timeseries charts have time going backwards. going lower to higher is certainly the standard way to do it. for eg: http://data.jobsintech.io/companies/google-inc

Any way for us to run `select` queries ourselves?

I haven't planned it but why not. I don't really know how to do it though.

Kudos my friend.

Great work mate

Reply on news.ycombinator.com