Show HN: Hacker News job trends | Hacker News Reader

Show HN: Hacker News job trends

github.com

371 pointstimqian92 comments

Nice work. I've been tracking this data for about 5 years on https://hntrends.com and enjoy seeing others come up with new tools and ways to analyze the data.

This is awesome! :)

Thanks! Glad it's useful.

I tried adding it to feedbin but it can't find any feed. Is it possible to subscribe to it with a RSS reader?

A RSS feed has been on my todo list, will get that added.

Nice, thanks!

This is absolutely incredible!

Thank you!

I was aware that Ruby On Rails had suffered a slow decline, and it was good to see some actual evidence of that.

Rails is definitely off of its peak, but has been pretty stable around the 10% level (10% of postings mention it) for the past year.

This is sweet. Thanks so much for sharing!

Just want to warn others that this didn't display properly in my mobile browser, seemingly due to whitespace/wrapping issues. It looks fine on my desktop.

It works for me if I set github to use the desktop version. I wish I knew of a way to always use the desktop version on my phone.

I really don’t mind most mobile sites. What I really hate that i wish i could disable is Amp from Google. Not only do they track your history without your permission with no opt out but they reduce the functionality of websites that i want to look at. They say the reduce the amount of data i use but when i have to load the website i’m visiting twice because i reload the page without amp, i’m using more data.

On Safari on iOS, long press the reload icon in the url bar to bring up a menu to request the desktop version or reload the page with content blockers disabled.

In this case, it does fix the formatting of the graphs.

I use a script to get an idea of demand in relation to others

You can grep for the fields you are interested to have an idea. This is the data I grabbed on the 2nd around 14:00EST

     79 python
     71 frontend
     63 backend
     62 web
     54 aws
     39 fullstack
     35 rails
     34 devops
     33 ruby
     33 mobile
     33 docker
     32 react native
     31 javascript
     30 c
     27 android
     26 linux
     25 scale
     21 security
     21 api
     18 nodejs
     15 postgresql
     14 django
     12 node
     12 graphql
     12 angular
     11 golang

I got kind of jumbled text from the command:

` 2040 204 ]]]]reply 46 ]]]]]]reply 8 ]]]] 6 ]]]]what]we]offer 6 ]]][]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]] 5 ]]]]requirements 3 ]]]]qualifications 3 ]]]]keyvalues 2 ]]]]you 2 ]]]]would]you]like]to]run]oracle]plsql]on]a]postgresql]database]you]can]with`

There's been many a discussion here about companies holding out for the perfect candidate, curious to see how many jobs go unfilled for how long.

I got a new remote gig recently. Because I'm going to talk about recruitment process and I don't think it's relevant, I won't name my new employer here.

My process had over a dozen stages, mostly 1+ hour video calls, one of them a 2-hour tech test. It took several months to complete.

I wasn't in a rush (I actually walked away from it for about a month in the middle for personal reasons), and they were being a little cagey perhaps because it was a senior-ish hire into tech (Principal), and picking the wrong person at that level can do serious damage to a tech culture. I appreciated they took it that seriously.

They then were prepared to wait for my notice on my existing role, which was 3 months (negotiated down to half that in the end).

That shows commitment on both sides.

This is going to be a stronger trend. There are a lot of people out there with a CV that says "senior developer" on it, asking for top-drawer contractor rates, feeling a little entitled because they know some React or Go, so it makes sense to take your time and really, really kick the tires.

With a remote workforce, able to hire anybody in any country, it makes sense to really find the best you can: they can make or break your business. Makes sense to me.

There is research - but I can't place where/when I read it right now - that says you should decide how many people you're prepared to interview, then discard your first third of that number. Then hire the first person you meet who is better than the best candidate in the first third.

It's actually 1/e ~ 36.7%, the solution to the mathematical problem of how can you find the best sample in a sequence, if you cannot go "back" and pick one of the previous ones. As long as you can ask people to "wait a bit" this is not the best solution.

For those interested in why, you can look for info on the "secretary problem":

https://en.wikipedia.org/wiki/Secretary_problem

A dozen stages? Holy crap. I guess they weren't worried about you considering other offers.

I made it clear early on I had dismissed approaches from about 10-15 other companies, and that they were my first preference, and they made it clear they were going to take their time on this one. We're all happy.

Personally taking so long to hire someone would be a big red flag.

I agree that hiring someone wrong can be damaging, but you can usually screen out obvious incompetence pretty quickly for any level real fast in onsites.

If I was the candidate I would assume they have a culture of waiting for perfection before shipping tomorrow rather than shipping something OK today and that is not the culture I want to be a part of.

Please note that I'm only detailing what I personally would infer from their decision. They may very well have a totally different culture.

The research problem you're thinking of is known as "The Secretary Problem" and Wikipedia has pretty comprehensive coverage on it (spoiler: it's N/e). https://en.wikipedia.org/wiki/Secretary_problem

they were being a little cagey perhaps because it was a senior-ish hire into tech (Principal)

Slightly off topic perhaps, but it's been my experience that these sorts of titles are meaningful and comparable only within a company. Across companies they don't mean much. A "principal" engineer at one company may not even be a senior engineer at another, or may be comparable to the CTO at another.

The terms are probably closer to being comparable within similar companies. e.g. the FAANG companies might have organically developed similar nomenclature to each other. But in general I find the terms are near-useless.

I wonder if others have the same impression. It would be nice if we had some way to calibrate positions across companies.

check out https://www.levels.fyi/

It reminded me of the joke HR rule about hiring candidates.

Take the top half of the resumes and throw them away because you don't want to hire unlucky people.

Were they offering significantly above market value? If not, I wouldn’t have been willing to go through all of the hoops.

They were offering San Francisco money, and I don't live in San Francisco. So, yes.

Yeah in that case with the opportunity to work remote and live in a cheaper part of the country, it would be well worth the extra effort on my part and I can understand the caution on theirs.

In the end hiring is a lottery for both sides. It is like finding a good marriage in a speed date.

It would probably hard to get accurate data on that, because a single job posting does not necessarily represent a single position. It represents a role that can be filled multiple times.

For example, basically every company is always hiring for a "Software Engineer" role. No company is publishing a new job description every time they hire a new Software Engineer.

I keep making the same posting but I've hired ~4 people and I'm still growing. One posting equals one job is a bad assumption.

Good to know. I have a half-finished project I was going to snarkily call “Who’s Not Hiring” that parses through HN Who’s Hiring archives and lists the companies who post the same ad month after month despite the so-called robust hiring market.

Nice! If you analyze gaps, you could use it to identify companies experiencing consistent churn too, perpetually re-hiring their engineering org.

You'd also identify companies who are growing their teams slowly and adding more of the same role every N months.

Job titles and descriptions could help in distinguishing between new and replaced roles. You'd want to look for fairly identical postings.

I think you might be overestimating the consistency and reliability of how hiring is done. (For reference, I've worked as a hiring manager of software engineers for a few years.) I've seen departments just use the same description over and over for years, and share it between teams -- even when it's not even really correct at some point.

Practices vary wildly between company, and even between hiring managers.

You could scrape LinkedIn for people who had been at those orgs for a short bit of time. Frequent posts of the same job description + LinkedIn churn evidence = red flag.

Does anyone know linguistically more about this "many a <singular thing>" construction? I've seen many an instance of this and I find it curious

My dictionary just says "used distributively with a singular noun, with an or a prefixed to the noun," and gives the earliest known example as "moni ane eærde" from Layamon's Brut (~1205): https://books.google.com/books?id=UFZcAAAAcAAJ&pg=PA218&lpg=...

Really nice work. An idea: Could we couple this with Who wants to be hired threads and see where there is a talent deficit and surplus?

Wanted to see more so I installed it. Just throws a "ENOENT no such file or directory, scandir '...\node_modules\hacker-job-trends\assets\rowContents'"

Nice graphs though. Source of that is https://github.com/kroitor/asciichart

I get get same error - though haven't really used npm before.

I’ve seen node or npm version differences cause this kind of error before. Not the OP but maybe try with the latest?

Uneditable, but obviously this was not the issue. Disregard. :)

As I mentioned elsewhere, the graphs were a disaster on mobile for me. I'd prefer if they were rendered into PNGs and inlined in the future, perhaps using something like ChartJS[1].

(I still appreciated and upvoted this project, though. I know nothing is perfect when it's first released.)

1. https://github.com/vmpowerio/chartjs-node/

It's a command line tool, who cares if it works on mobile.

It was posted on a website which, if it's a typical news site, gets at least 40% of its traffic from mobile devices.

I understand your point, but at least the readme could have images as a demo.

Anyway, it doesn't matter that much. Clearly we all like the tool regardless.

The assets are missing [1], just clone the github repo and move the offending directory to '...\node_modules\hacker-job-trends\assets\rowContents'

[1] https://github.com/timqian/hacker-job-trends/tree/master/ass...

Thanks for reporting this, the reason is I ignored the rowContents to npm registry, it is fixed now.

Nicely done! I really like the example graphs you give.

I'm sure this crowd would appreciate seeing an example graph for Kotlin, Scala, or Rust as well. :-)

While we're discussing hiring trends, what's the state of the "full stack" meme? Is it really realistic to expect a developer to be an expert in frontend, backend, and database design? They'd need to be an expert in JS + whatever the framework of the week is, CSS, some backend language, and SQL. Increasingly, companies are adding significant DevOps skills to this list, too.

I feel like it can be realistic if that person has about 5-10+ years of experience and a lot of time on their hands to keep up on frameworks. But a junior level full stack person, probably not.

Backend devs I find usually also do a fair bit of database work, at least at a basic level, and are usually where the devops work comes in (migrations, upgrades, keeping the system running). As experience grows, I think most people at least get a sprinkling of these different fields, even if they didn't intend to.

That sounds about right, and reflects my experience and expectations as a backend developer well.

Not taking anything away from this nice job and at the danger of saying something obvious I'd like to note that the HN job board is not representative of the whole IT market. It's a corner, an interesting one but when it comes to the question what skills provide the best overall career outlook in the long term for a developer the trends on HN are not necessarily the best predictor.

Nice work, interesting trends. The monospace font was a mess on Linux at least, had to deactivate the font-family css and it picked up my standard font (source code pro) and all was right again.

If your goal was to be helpful, you should really be more specific than just "linux". It works fine on Firefox / Ubuntu Xenial for me.

In any event, I'd suspect it was more likely for the problem to be in the browser rather than than the kernel...

If your goal is to be helpful a little less pedantry would be in order.

Firefox / Ubuntu Bionic, probably related to font choice or font-rendering library.

When using multiple keyword with the + and - signs, it graphs the sum of all posts requiring one technology in addition to all posts requiring the other technology.

It'd be great if there was a way to graph sum of all posts that require both (or more) technologies. I have tried with spaces and comas etc... (eg: 'rust, python') but it doesn't work well as you have to guess exact matches (so 'rust, python' is a different result than 'python, rust').

You can also end up with results like some months having a negative number of posts. I think the intuitive meaning of any arithmetic operators here would be set operations.

Has someone already made a map with the countries where the jobs are being offered? If not, I might do so.

I'm currently trying to find a job in New Zealand, Australia, or Canada.

Yes, here it is: https://whoishiring.io/

HN's "Who is Hiring" jobs only filter https://whoishiring.io/search/36.6845/-92.8938/4?source=hn

The graph part is pretty cool. Is it doable to use this graph for any kind of data? Or maybe someone know a tool to trace similar graphs into a terminal?

Looks like it's using asciichart: https://www.npmjs.com/package/asciichart

Thank you very much!

Interestingly named package given that it's clearly using quite a few non-ascii characters which will break non-unicode aware consoles. Also that the description, Node page, and github repo list different licences.

"ascii" these days is so often used as a synonym for "text only" that I suspect a fair share of the people using it as a word does not know what it stands for or what it originally meant.

Everything in there looks ascii to me, just using the charting characters in the high end of the table.

"ASCII" is limited to characters 0 - 127

https://en.wikipedia.org/wiki/ASCII

This looks horrendous on mobile. I can do Request Desktop Site on my mobile browser and it looks better but not perfectly aligned still.

This is very cool. Is there a way to adjust search results for total number of posts? By searching for `hjt 'a'`, I can get an idea of how many total posts there are by month and approximate adjustments in my head, but it would be nice to directly scale results.

Would love to play with the dataset in the browser. I assume it's small enough to load all comments into an array and then let the user run their own queries and ouput the result into a nice chart library. That would be fun!

Yeah, row content is about 16mb, This is exactly what I will do in the next step

Good job.

Why did you prefer using the HTML format instead of HN API? https://github.com/HackerNews/API

Perhaps because you can request a page of items at a time? For something like this an even better alternative might be the BigQuery dataset: https://cloud.google.com/bigquery/public-data/hacker-news.

Well. Expecting an article about job trends, with good old text and maybe some new cool info-graphics, and after clicking the link discovering a file/archive directory first and them some vintage looking graphics... is weird.

Thank you for your work. Really helpful.

Fantastic work! I will use this. :)

Nice tool, I wonder if trends can be used for marketing and recruitment?

I'm not thinking for marketing or recruitment but for educational institutions trying to push certain courses it could work.

q a

A data dump would have been much nicer, I don't want to have to install a package to basically run grep on a dataset. The tool should only be for updating the dataset.

This attitude I see all the time on HN absolutely irks me (e.g. the whole 'A graph of programming languages connected through compilers' thread). Too often people post something fun and cool they built and readers complain about some random feature they think would make it better. It's not like OPs goal was to solve every little problem with graphing terms in HN Job threads. There should really be a rule against this sort of critique.

Its one thing to make a feature suggestion to be helpful to the author and a completely different thing to provide armchair critique like this.

The tool they wrote is to do the job they wanted to do. If you want something else, build it. If it doesn't satisfy your needs or you're too lazy to fork it, move on.

With that said, the data (text and HTML) is literally in the assets dir of the repository which should make it easier for you to build your version of the tool. Did you bother looking?

It feels shameful when you see it directed at others, but then if you post your own stuff you don't want the feeling that the community is pulling punches just to play nice.

Fortunately most of these things are open source, so there's always the 'well submit a PR' response. It's been so long since I slung HTML that I am fairly certain if I attempted it for a Show HN, folks would be so distracted by how bad it is they wouldn't pay attention to the utility of it.

I did actually bother to look in src, but for some reason didn't check out assets. Now that you mention it, it seems quite obvious, not sure how I missed it.

HN comments are in bigquery. It's a simple query to extract that data, I just wrote it: https://bigquery.cloud.google.com/savedquery/1008801496706:d... . It seems to be outdated tho...

what's outdated? The dataset?

Never mind - see other comment by minimaxir. You have to use table bigquery-public-data:hacker_news.full rather than comments for up-to-date results.

The `full` table has comments up to recently (May 19th). You just have to add a filter on `type = "comment"`.

https://bigquery.cloud.google.com/table/bigquery-public-data...

Reply on news.ycombinator.com