Show HN: Analysis of 2018 Hacker News “Who Is Hiring” Job Posts
letstalkalgorithms.com
letstalkalgorithms.com
I think the reason Scala seems surprisingly popular is that is was simple string matching so a post that said "We are looking for someone who can build scalable infrastructure" would count as one mention of "scala".
Could be wrong though.
Also see: https://news.ycombinator.com/reply?id=18725655&goto=item%3Fi...
I would be pretty surprised to see these two so much more commonly requested than Javascript.
Honestly, it is an insult to call good data preprocessing skills grunt work.
It may not be as attractive, but requires a lot of understanding of the nature of data and way in which patterns present themselves.
For what it's worth, I don't really see this as a problem in and of itself. Of course, if you want to do data science in the real world, you need to learn about data cleaning, manipulation, and warehousing in addition to the "pure" data science process that begins with tidy data. But it's good that tutorials segment these out, since not everyone needs to learn both at the same time. Anecdotally, I had plenty of real-world experience dealing with messy data by the time I started learning data science.
As an aside, my impression is that technical assessments for data science positions tend to underemphasize data cleaning and manipulation. Granted, they're still really time-consuming as it is, but there's probably some room for optimization there.
True, although ideally the data is cleaned/structured before it hits a data warehouse and the data scientist starts working with it. It's an iterative process (DS finds messy data, flags the problem upstream, repeat).
I do wish there were more tutorials with messy data; the ones I make I deliberately try to use complex datasets and highly relational ones, although as a result the tutorials are more complicated and more imposing for beginners.
> As an aside, my impression is that technical assessments for data science positions tend to underemphasize data cleaning and manipulation.
When I was interviewing for DS jobs, I got a lot of the implement-binary-search-on-a-whiteboard questions which annoyed the hell out of me. But for the takehome assignments, they required some data ETL, which I felt was more representative. (some assignments had deliberately flawed datapoints the user was expected to identify and remove while not explicitly being told to do so)
They're just calling it like it is.
(lightheartedly; I'm not like disgusted over here :-) )
Edit: trivial isn't the right word, it's obviously an important step in the process but quite different from what you are really paying a "Data Scientist/ML Engineer/etc" to do.
It's not that putting the sculptor on the task would be a waste, nearly as much as that the sculptor doesn't possess any of the necessary skills.
How many people know how to find a source of marble, run a mine, and sculpt a great work of art? Roughly nil, but it's how many organizations seem to hire.
It's literally what the phrase means.
Market research is all about getting different kinds of data and massaging it into queryable and productizable form (reports).
When can you start?
https://hnprofile.com/compare?search=scala,python,rust
Not the only issue I see, for instance my company posted for “machine learning engineers”, yet that isn’t in the results at all.
I think they are doing regex search in the strings, as opposed to identifying the words and doing comparisons
Because there are a multitude of possibilities.
/asp\.net|dotnet|c#|VB.net|f#|xamarin/iThis analysis has the Austin as one of the top 10 locations, and I'm curious whether or not that is skewing the numbers. There are a steady stream of jobs actually in Austin, so either seems possible.
Edit: I fixed the false positives using a word boundary on regex "\\bword\\b". The data now seems little sane, will keep looking for any false positives. updated the data and regenerated the chart using new data.
I get that you're trying to be complete, but if it's not a truly remote position, it's not remote. put nothing about remote in the position or at the very most "REMOTE: NO". Working from home 10% of the time but needing to be in a SF office the rest of the time is not a remote position; it's the flexibility I'd expect from most jobs in this industry.
"Sorry, no couples!"
And same would go for visa. "We do not sponsor VISA"
I really think you're going to have to do some sort of sentiment analysis, either via human or machine.
Anyway, I've personally seen (and interviewed for) jobs that were advertising Scala (or plans to move to Scala, or that Scala experience is a plus) but actually meant there was one proof-of-concept project in Scala, 18 months ago, nobody ever touches it nowadays, the person who wrote it is long gone and by the way, our legacy JDK5 application is on fire, do you mind fixing that for the rest of your life?
https://www.hntrends.com/2018/nov-react-still-top-containers...
Hype-driven development is a problem, and hype-driven recruitment another one. Just be a good java developer. Maybe push for Kotlin slowly. Don't pursue Scala for the language's sake.
See also: blockchain, IoT, etc. I've seen a job advert that basically bolted those onto the not well hidden "we're just looking for decent java developers" job description.
I know it's just one part of the analysis, but the skills list appears to be be pretty far off of what I've been seeing. React gets over 200 a month by itself. Are you capturing all pages of the postings in each month's thread? Here's the data (counts) I have through November, broken down by month and term - https://www.hntrends.com/data/data-20181101.js
I meet with clients a couple times a month and I am on the phone, slack or webex/ringcentral/google hangouts etc.. a few hours a week.
I am _sooooo_ much more productive now (with fewer distractions) that I work on average 500 hours a year less than I used to. That doesn't even begin to factor in the quality of life issues. I now drive 8000km per year versus over 20,000km per year.
Some jobs, some roles, yes, you need to physically go somewhere. For me, for what I do? There is no benefit.
I'm a big believer in serendipity. To use a machine learning example, your algorithm needs to have some temperature.[1] Sometimes you want to sacrifice your queen in order to checkmate your opponent in 5 moves. In this situation, that might mean sacrificing on productivity during a project in order to meet with the client more frequently in person, allowing you to develop a long lasting relationship.
I'm sure that things come up in those physical meetings that don't come up during phone calls or Slack for a variety of reasons. In my opinion, if you met with clients a dozen times a month, instead of a couple, you would not be as productive, but you would drastically improve your relationship with the clients.
Loyalty is a currency like any other, it can earned and spent. You can't quantify the value of a relationship the same way you can quantify productivity, but your improved empathy and sympathy to your clients problems will improve your performance. You might also find that your clients trust you more, and give you more freedom and time to find solutions to their problems. Finally, developing relations will pay off in your professional and personal life down the road, long after you have finished working on the current project.
Of course, if you're optimizing for work-life balance, or spending less time on the road, then this can all be ignored. Take the necessary steps to achieve your desired lifestyle. If your goal is to promote growth, build infrastructure, and deliver value, I believe you lose something by going remote.
Personally I think the best system is a combination of Monday and/or Friday remote, with the rest of the days in a physical location. This allows employees to enjoy parts of the remote lifestyle, while still keeping many of the benefits of meeting in person.
1) https://www.quora.com/What-is-the-temperature-parameter-in-d...
Plus, free lunch.
I personally am commuting close to 2 hours each way, so 4 hours total, because the job market is much stronger in NYC then my immediate (30-45 minute) area. If a job is open around here the salaries are almost 30-40% lower than NYC despite our COL still being high.
In regards to your commute, wouldn't a 40% lower salary be worth 4 extra hours everyday? I guess it depends on the math but unless you're being compensated for the commute(in which case it's just work on the train/metro) that seems like an incredible sacrifice for a bigger paycheck. At what point is your time worth it?
You mean managers and owners getting to oversee "their people".
As for the commute, I've worked out a flexible work schedule that lets me be home a few days but as I stated I'm in a higher COL state (NJ). So a 30-40% reduction in pay ends up being a fairly big change in QOL. I've learned to make use of the time on the train by engaging in things I enjoy (video games, podcasts) which may be harder at home. Moving closer also doesn't help because A. it gets even more expensive B. QOL drops due to higher density areas which makes it harder to ride bikes, garden, etc.
We've accepted the choice we make to be where we are because of what we get from our life outside of work, but it doesn't mean I can't hope for better remote possibilities.
Secondly there's something like a "no true Scotsman" vibe about the implication that you can't trust people who prefer remote work to comment on their productivity like you can people who are onsite. That may not be the right fallacy, but there's a fallacy in there somewhere.
Thirdly, nobody should be working with kids crying. God invented doors for just this reason. If you can't make a quieter space at home than you can at work, you're either pulling in some serious perks on the job with your private, sound insulated office or you don't have the fundamental amenities to work at home, it's not the nature of working remote, it's showing up to work unprepared that's holding you back.
It was kind of stupid to come into work when this rule is in effect.
One option is to re-create that talk. At a past job we would often all fuck around before/after standup calls for a bit, but sometimes that fucking around became work and solved problems. Other times we would just call each other either one-on-one or in small groups just to shoot the shit, and that can recreate that same feeling.
A big part of that is getting over the idea that "calls" are somehow different than walking over to someone's desk, you wouldn't hesitate most of the time to walk over to a coworker and start chatting, but most people hesitate to call someone on slack. At a past job that hesitation wasn't there because the culture embraced it, and suddenly we had our watercooler talk back, just over video calls.
One remote first company I interviewed with told me all their daily calls were video because they wanted to make sure people still felt human. And I think it's such a crucial piece in all this because people have been accustomed to just doing audio only which only further creates a sense of loneliness.
It's worth it though, and when I'm on-site I get people talking to me that I've never met who recognize me from somewhere I was on a screen.
So you are right something is lost. Skype, jira, slack will not convey the feeling that there is another person on the other side of line. You will not see coworkers getting sick, going through hardships in life. If you see someone you can tell he had bad night sleep that is why he is upset. You get the watercooler conversation that other guy wife is annoying... You don't care it should not affect quality of work but no one is robot, via electronic communication everything seems so perfect... Then you expect people to be perfect, and they expect from you to be perfect, then you get upset, but you just have flu and cannot focus really...
And as I stated you can replace this in other ways. Because you're home you may end up at a gym or coffee shop or group bike rides more often. That gives you a different form of social interaction in the day to day to replace "water cooler" talk.
Of course it's not 100% that way, but my own experience has seen it play out a few times that way.
There is a middle ground. One can open up small offices in small cities. For example, instead of 1000 people in a city of 5,000,000 people, it could be 20 people in 50 cities that have 100,000 people. Each site gets a VPN connection, with the hardware physically secured in a commercial building.
The COL goes way down. The commute becomes tiny for most people. I'm in that situation, and my commute is 3 minutes if I use a car.
But I don't want to see it if the market doesn't justify it. I think today there is something beneficial to having people working together in an office, but I'd prefer to be proven wrong by some new remote-work management style (or something). And software development is one of the more ideal use cases for remote work, so if it's not expanding for us, it's less likely to expand in other industries.
Python is deeply troubling. It is a regression from FORTRAN and COBOL. Long ago, we invented compile-time type checking. The benefits for software quality have been enormous. There isn't really a downside here, as there would be with the performance loss of garbage collection or bounds checking. Shaking out lots of bugs before even attempting to test the software is a wonderful advance that we made half a century ago. Python's incompatibility with compile-time optimization is also horrifying. The situation is so extreme that you can't even make a decent-performing JIT.
Both systems of type checking have existed as long as... Programming languages have existed, essentially and, sadly, so have the endless comparisons and flamewars.
it felt like a revelation, and almost disappointment that there is hardly any innovation in newer languages.
Python does not have static typing as a built-in part of the language. It has had the ability to annotate arguments and returns of functions/methods since 3.0, a standard-library module containing helpful code to use this for type hints since 3.5, and the ability to annotate variables since 3.6. Annotations are a completely optional feature and no built-in part of Python will check these annotations or analyze code for correctness in advance of execution; there are third-party tools to do this, if you want it.
Also, Python most certainly does compile -- the CPython interpreter is a virtual machine which runs bytecode, and Python source code is compiled to bytecode for that VM. There isn't a requirement to run a completely separate standalone Python compiler ahead-of-time to generate the bytecode (if bytecode isn't available, Python will compile source to bytecode on a per-module basis as those modules are loaded), but that doesn't mean it isn't compiled.
As to "strong" typing:
"Strong" typing is a term that's only vaguely defined, but most commonly refers to whether a language will implicitly coerce/cast values of incompatible types in order to make an operation succeed. Consider this code:
a = 1
b = "2"
c = a + b
Static typing refers to a situation where both names and values have types, and where all attempts at binding must involve names and values of compatible types. For example, in Java:
int a = 3;
You can remember this easily by considering why the name "static" is used: it's because you can perform checks of name/value types and bindings statically, without needing to run the code to determine types. In some languages (like Java) this is accomplished by requiring all names to be explicitly annotated with their types; in others the types will usually be inferred automatically from usage, with the option to annotate when desired or to resolve ambiguity.
--
¹ Yes, yes, I know someone on HN is going to suffer a terrible career-ending injury from how fast his knee jerked at that and possibly from breaking his wrists in his rush to post "Well actually there may be a type defined somewhere that's a union of string and number, so how dare you say that's an error when you don't know if someone might be using such a type!" My advice is not to be the type of person who suffers severe injuries due to such an obsessive need to nit-pick, because reasonable/charitable readers will correctly understand the example with no difficulty.
Of everyone who asked for work or applied for work via these posts - how many secured a position?
I've long given up on these posts trying to find freelance work or other permanent work. There's to much competition (here) to stand out from the flood of replies.
My advice: skip this site and weworkremotely.com. Both are complete wastes of time based on the responses.
If you’re in IT, your best bet is LinkedIn (for a referral) or Careerbuilder/Dice/Indeed. At least those will result in face to face interviews.
Also find companies you like and apply directly on their website.
* SF is center of the tech world. Next biggest is just around half of its size. (not withstanding HN bias).
* Approx 25% jobs allow visa sponsorship
* HTML, Python, .Net overwhelmingly dominates everything else.
* Reduced interest in databases, SQL, Obj-C, Java
* No TensorFlow or PyTorch in demand
* 90% jobs in development/software engineering, 10% in management and misc.
The article lists SF as having 2922 jobs and New York 1746 jobs
I always assumed it was used heavily in enterprise but never that much it's would somehow , it's also a popular stack in the Silicon Valley and NY. That said London and UK are well know to be MS Stack users.
Can some Engineers from SF or NY or voice their opinions on this ?
"Engineering Manager": 271,
"Principal Software Engineer": 25,
"Product Designer": 243,
"Product Manager": 425,
"Program Manager": 35,
"QA": 353,
"Senior Software Engineer": 818,
"Software Engineer": 2975,
"Staff Software Engineer": 10
https://news.ycombinator.com/item?id=18662978
https://news.ycombinator.com/item?id=18590124
There's probably more in December alone that the regex missed.
I'm very skeptical that there are 18 new jobs on the island of Jinja, and that instead the analysis is picking up the Jinja templating language
Edit: same thing for "cambridge" in the same location as Jinja Island. Obviously probably refers to Cambridge MA, where Tufts/Harvard/MIT are all nearby, so tons of companies will be there.
Can you better express how you extracted the location data?
Also I don't think this is de-duplicating, and many of the Melbourne FL ads are repeated.
I think it's so high because "excel" is also a verb so a lot of job posts will say things like "We are looking for people who excel at what they do" and this was counted in the analysis.
Changing the search to be case sensitive might help here.
"scalable solution"
"excellent benefits"
I don't know but maybe the author it's looking simply for "excel" and "scala" in the text without taking in account surrounding space. So:
- "scala" would match "scalable" and "scalability"
- "excel" would match "excellent"PS: I use Kotlin/Java for web development.
It is sad to me.
Skimming a few of the posts, there are maybe 15+ references to C++ for post, and 1 for Julia at most.
The detection algorithm seems to be a bit iffy.
Pretty sure that a string like "c\+\+" was there before.
It seems that in the new version the old patterns have been commented out, the search method was changed and c++ removed altogheter:
https://github.com/venkateshthallam/hn-whoishiring-analysis/...
I know Python can be used in practically anything, but what do you keep seeing companies ask for that is a recurring need with Python?
Our Data Scientist was able to pickup Django pretty quickly because of his experience with Python. And yea, actually someone at my company who was doing a bit of scraping on the side was able to pick up Django pretty easily as well. And last night, I chatted with my buddy who works at a big tech company and they use Python for database intrusion analysis.
I don't know the Ruby ecosystem very well, but it seems the biggest application of Ruby is Rails. I'm curious to hear what other ways people use Ruby, so please share.
I also use Chef to setup my local environment which is wonderful. When you are learning how to really dig deep into your text editor and writing your own custom snippets (and commands, and keybindings) for react. You don't how important it is to have your local env, your dotfiles and atom config automated and can be synced up with a single command no matter what computer you are on.
This, with private git repos and syncthing is a wonderful thing.