Show HN: my weekend project – browse jobs posted to HN
hackernewsjob.com
I put this together to help browsing the 'ask HN: who's hiring' threads. It's nothing fancy, but it helped find a couple leads. Maybe it can help you too!
hackernewsjob.com
Great job! Much better than ctrl-f-ing my way through postings [just like you were doing].
To answer your first question:
I'm using a custom scraper to get the results. I request the job pages through a random http proxy every so often, parse them, and update the cached json of all the comments. Getting a reliable, free, machine-friendly list of http proxies is harder than I thought. It was automatic, but the proxy provider I was using went down (doh!) so now I manually pull one from a list and trigger the scrape.
On the server I'm using a tiny express app (node.js) + redis to store hidden posts per use based on a cookie.
On the browser just a little bit of twitter boostrap + jquery + some custom UI helpers to generate the page. The searching and hide/show is done within the browser itself. I try to involve the server as little as possible.
To answer your second question:
It will update as soon as I kick off another scrape. I hope to re-enable the automation on this soon so it will only lag 10-15 minutes behind the actual page. Right now it could be quite a while (up to 12 hours).
Yeah! ctrl-f was tough. Especially as the comments spanned multiple pages and weren't sortable by date.
Is it better to have an xml feed and load that instead of a JSON file? I wrote a scraper that dumped university courses at my school to a JSON file. I would load that JSON file and load the elements to a table. Mine was pretty dang slow.
But your site holds up fine.
Here's my site if you're at all interested: http://lo.leet.la/jola/bootstrap/docs/sunny.html#
Relvant JS code: http://lo.leet.la/jola/bootstrap/docs/assets/js/sunny.js
Edit: And the YC job postings are located at http://news.ycombinator.com/jobs
There are absolutely tons in NY.