Show HN: JobLens – AI-powered job search for 'Who Is Hiring' | Hacker News Reader

Show HN: JobLens – AI-powered job search for 'Who Is Hiring'

kadoa.com

108 pointshubraumhugo18 comments

There are existing HN job aggregators, but I thought we could take it a step further. Inspired by an insightful comment on a previous thread (https://news.ycombinator.com/item?id=36163021), I built a tool that aggregates job postings and intelligently categorizes them based on user-specific preferences:

* Country and remote work preferences

* Employer type (e.g., startup, corporation, government)

* Industry

* Technologies used

* Role type (developer, architect, product owner, etc.)

* Salary range (where available)

One of the superpowers of LLMs is reformatting information from any format X to any other format Y. We leverage this to map all the unstructured job postings into the same unified structure. The new GPT functions feature and the extended context windows are really helpful for this. Instead of having to build a custom NER pipeline, it works very well with GPT out-of-the box.

One challenge is keeping the filters consistent and merging of duplicates. Embeddings help with that.

What's next:

* Integrate additional sources. We can generate web scrapers and data processing steps on the fly that extract and transform data into the same structure.

* Add location distance filters.

* Expand beyond jobs to monitor personalized data like events or real estate. Imagine using AI to rate local events from multiple sources based on your preferences, considering factors like your interests and distance from home.

* Smaller improvements based on your feedback :)

Open articleView on HN

JobLens is cool! This past month we had the same idea -- it's been a fun project: https://hnjobs.u-turn.dev

ChatGPT does an incredible job parsing, but then lots of effort goes into normalizing and deduping each field. Long story short, your results look quite good to me!

Your project looks quite impressive as well, especially the extracted URLs to apply and the candidate profiles, didn't get that far yet :) Automating tedious work like data extraction and transformation is a great use case for LLMs.

Not sure if you found this as well, but gpt-3.5-turbo-016 does a poor job following instructions other than parsing. So, to work around this, we prompt gpt-3.5-turbo with the rules we want applied say an extracted field and then go back to gpt-3.5-turbo-016 to parse with chatgpt functions.

Bottom line, every single post requires approximately 10 different prompts to refine the extraction.

yeah I had the same experience, divide and conquer works better than trying to do everything with a single prompt. Are you interested in comparing notes? Feel free to ping me via the email in my profile :)

Hi, just a thought, I backed out as soon as I saw I had to pick from a giant list of industries. Most people don't have rigid criteria like that, and if the do, a conventional search is easy. Conversational AI has a potential to more naturally arrive at interesting jobs, either through asking for a freeform description of what you want, or maybe a back and forth discussion.

I got turned off by that as well, but I submitted the form anyway, and it loads correctly. I guess no fill in means *

You're right, I'll improve this.

Wow this is amazing. This is more or less what I was trying to build https://hnresumetojobs.com to look like, I am reconsidering whether or not to continue now.

I tried using the OpenAI functions API to do structured text extraction, but found that it would hallunicate a lot of things. Do you guys have that problem? How did you guys go about solving that?

Having the 'unchecked' visual actually have a checkmark threw me off. I was thinking, oh goodness, I have to deselect all of these.

Ask ChatGPT to improve your prompt to stop it hallucinating. I’ve had a lot of wins doing that.

Looks like it's getting some fields wrong. For examples, "Global" location is matching other text not related to the job location.

Still, a good example of OpenAI's functions. I was using direct prompt for YAML parsing before and migrated to functions for my job board https://aijobnetwork.com/

I'm also using OpenAI's embeddings for recommendations: https://medium.com/@mauricio/how-to-add-recommendations-to-a...

Need a category for design. Right now all design jobs are scattered around, mostly in product management. Also as others have mentioned, having to make all these decisions before searching is hard to get through without abandoning.

This is cool. I'm curious about this part:

  We leverage this to map all the unstructured job postings into the same unified structure. The new GPT functions feature and the extended context windows are really helpful for this.

Does this mean that, rather than using the gpt-3.5-turbo chat API, and using a system prompt that describes the output format you want (JSON?), you instead describe your output format in the same way you would if you were going to call a function?

Yes, you can essentially use GPT functions to reliably output JSON based on a predefined schema. See https://github.com/openai/openai-cookbook/blob/main/examples...

I was going to build something similar for funsies but now I don't have to :)

I'm curious the page mentions not being able to scrape sites that require a login but then I saw LinkedIn as an example. Will LinkedIn eventually be supported but require a login or something?

Site is down for me, maybe the HN hug of death? Anyone else seeing this?

Nice! Is there something similar for freelancing gigs?

All I want is a list of jobs that involve functional programming.

Reply on news.ycombinator.com