I scraped 1.6M jobs with GPT4o-mini

I was sick and tired of how LinkedIn & Indeed are contaminated with ghost jobs and 3rd party offshore agencies, making it nearly impossible to navigate.

I realized that most companies also post jobs directly on their websites. Until recently, there was no way to scrape them at scale because each job posting has different structure and format. After playing with ChatGPT's API, I realized that you can effectively dump raw job descriptions and ask it to give you formatted information back in JSON (ex salary, yoe, etc). I used this technique to scrape 1.6 million jobs (with over 325k remote jobs) and built powerful filters. I made it publicly available here in case your'e interested (hiring.cafe).

Pro tips:

* You can select multiple job titles and job functions (and even exclude them) under "Job Filters"

* Filter out or restrict to particular industries and sectors (Company -> Industry/Keywords)

* Select IC vs Management roles, and for each option you can select your desired YOE

... and much more

Please let me know how I can improve it on r/hiringcafe!