Show HN: A real-time list of open tech roles

Hey HN! Quick TL; DR: I set up scrapers for the websites of ~20k tech companies, fed the results through GPT, spent some sleepless nights building a web app, and voilá.

I started this after getting fed up with how slow LinkedIn felt. I finally got it working and easy to use, so I wanted to share it with you guys.

The job list is free. But I’m currently paying a lot of money to OpenAI for all the data I’m passing through GPT, so I may end up eventually charging for some advanced features, or putting ads on it or something.

Please let me know what you think!

I’m actively developing this so I’m happy to add requested features.

Some technical details you may be curious about:

- I began gathering the career pages for each company manually, but I ended up hiring a few Filipino contractors to speed up the process.

- I run the web scrapers once per day.

- I do the web scraping with Node.js. I use cheerio for basic web pages, and puppeteer for more complicated ones.

- My OpenAI API dashboard tells me I process 7 million characters per day.

- The list doesn’t encompass every single tech company (yet). I have most of the major ones (~20k), but the process of adding new companies is pretty labour intensive so it’s still ongoing.

- The web app is built with React, and I got the infinite scrolling to be so smooth using a great library called React Query.

For feature requests, please leave a comment here or email me at dorfmanjames@gmail.com