Show HN: A Machine Learning Model for Salary Estimation | Hacker News Reader

Show HN: A Machine Learning Model for Salary Estimation

elaineou.com

83 pointselaineo31 comments

(1) this is awesome

(2) http://glassbowl.info/ is not leading for me

(3) If you did scrape LinkedIn, you probably violated their ToS, so you may not want to admit this publicly in case this goes anywhere :)

Came back here to suggest your point 3. I sense a cease and desist is on the way...

Can I plead ignorance? :X

You can always try I guess. Whether or not it works is another story :-p

I started using tor for this type of thing. It's very handy in that it generates a new circuit every 10 minutes or so and hence you get a new exit node.

Re #2 - sorry! I had to restart the server. Try again now :)

For me, it's now saying: "Oops! LinkedIn error. Please check your url."

Usually it goes away if you wait a little bit. I think they do some rate-limiting. Also they'll probably block requests from my server soon ;)

Could you get away with allowing folks to manually paste in the HTML of their own linkedin page, or perhaps upload a PDF version of their linked in page?

Are you scraping the page for input params? Would it be possible to just expose an API (or web forms that point there) to let folks put in those parameters manually?

oy, that would actually have been a much better way of doing it. I was trying to be clever and keep the UI simple, and totally didn't expect to hit rate limits so quickly.

I want to +1 this idea :)

Okay, I added a field to manually paste in the HTML! Thanks for the suggestion :)

I did something similar for my side project at: http://skill.report

To avoid the LinkedIn issue, I crawled job ads (about 10M uk ads over two years), and applied analysis to them. I love your approach - I'm going to have a play this weekend :)

LinkedIn has so many amazing datasets locked up in the platform. I was keen to see whether I could reproduce the skill graph that LinkedIn has, and apply it to other problems, such as valuing and comparing skill sets and knowledge from job titles alone.

Oops! LinkedIn error. Please check your url. But I'm pretty sure my URL is correct. It's the gray little URL underneath my profile picture? Similar to the example shown in the text input on your page?

Same

I've noted that app is expecting to start with "www." in the url.. which is not the case for my url.

But if you change that portion, LinkedIn will still find my profile, and the page seems to be working; as always YMMV

Sorry, it was LinkedIn's rate limiting :( try again!

It's dead, Jim.

The demo seems to be down, but if it works the way I think it does, it is a clever honeypot for scraping warm lead LinkedIn profiles.

It would be interesting to see if the model produces biased results by race, gender, age, etc.

That would actually be a really great idea. I wonder if it would be tough to get race/gender/age just from the linkedin profile.

You can probably get away with undergraduate degree graduation year as a rough proxy for age.

age(grad_year) ~= (2016 - grad_year) + 22

I'd love to talk more details with you sometime! We(PayScale) don't scrape linkedin, but we have been collecting, analyzing and predicting pay for over a decade.

My linkedin URL has a prefix "br." because I signed up when I still lived in Brazil. That makes Glassbowl crash and alert me the URL is invalid...

Mine has an "es." but if I remove the prefix it still works.

It's down for me

One way to train your model would be to allow people to tell you their current salary in exchange for the analysis. Then you get more accurate salary data. Of course people could lie, but that's probably no better than the glass door data anyway.

Getting an internal server error

can you reversely try to guess how much other people earn?

it fails for me :-(

Could LinkedIn sue this person?

I like this one, it lets you play hundreds of simulated games per second, and even over thousands of years you still don't win.

http://powerball-simulator.com

Reply on news.ycombinator.com