Hacker News Reader

Comment by carlmr | Hacker News Reader

<<Most damningly, he says the questions posed by HackerRank and similar tests aren’t necessarily relevant to the work actually done in finance technology roles: “They’re the sort of problems you’re posed at university. – There’s a focus on data structures and algorithms, but this is less relevant when your success as a developer is in building complex enterprise systems.”>>

Exactly on point. Also one gripe I have with hackerrank (I did some of the challenges to brush up on some language knowledge), is that a lot of the questions are poorly designed and from obvious amateurs.

I still think that neither whiteboarding nor hackerrank selects for good developers. In the end you get people that studied hard to solve a certain class of problem that you'll never need in the real world. It's wasting candidates time, and it still shows nothing, except that your candidates know something unrelated to the job.

View on HN

> similar tests aren’t necessarily relevant to the work actually done in finance technology roles

IMO, what's relevant for a role is the skill set most people currently doing that role have and that's not always a good thing.

I've been the first shouting on top of a roof that we shouldn't ask people to know stupid little algorithm trivias to get a job. But the narative for that quickly devolved to "data structures and algorithms are not important for day to day work". To which I honestly say bullshit. The industry is managing without it: between people who forgot everything from school, bootcamps, self taughts, and people with alternate degree, people who can do actual computer science on the job are a tiny, tiny minority. So the solutions those people come up with to day to day problems are considered a speciality (eg: "machine learning!", "data science!").

But if doing that shit was just second nature to everyone, how would it change the way we do things? What if category theory wasn't so scary? What if people weren't scared of threads? What if hashmaps were not magical black boxes people use because the person who wrote the code before them did?

The foundation is becoming pretty darn weak if you ask me.

I totally agree shit like HackerRank is poorly done. But I'm not sold that the skillset it tries to assess (poorly!) is that useless. We, as a community and an industry, essentially did everything we could to MAKE IT useless. That's different I think.

I consider it from a problem-solving perspective: I don't need to know the intricate details of dozens of sorting algorithms or data structures. What I need to be able to do is solve the problems that get thrown at me in daily work. So if I get assigned, say, a performance problem I might, after some research, come up with the idea to use a different data structure to solve it. Or something entirely else! Depends on the problem and where my research leads me.

As a software developer I see myself mainly as someone who solves problems to help others do their work better. So, while I agree that having a strong CS background is definitely a benefit, I don't think it matters that much if you can't get binary trees just right after not needing to implement them for years.

> So if I get assigned, say, a performance problem I might, after some research, come up with the idea to use a different data structure to solve it.

Yup. And my feel on that is that (assuming the perf issue is related to a core data structure), that -particular- perf issue shouldn't even have happened in the first place. It should have been obvious given the right background and you could have spent that time doing something less fundamental.

Things evolve. The scale of problems evolve. A data structure that was handling hundred of transactions, crumbles on hundred of thousands. Premature optimizations (and premature abstractions) are the root of all evil.

Corollary: A data structure that is instantiated million times in small sizes is different from one huge instance with million entries.

No argument there, but that's not what I'm arguing.

Do you have to reach out for google or a book to do a simple if/else statement? What about a for loop? Now what about the difference between a map and a set? Likely not, it's the basics used in any app.

Talking about premature optimization here implies there's some work to be done, things to think through, stuff that's not obvious. And that's totally true at a certain complexity level.

But what is "complex" and what is "obvious" isn't objective. It's purely a factor of what the "average software developer" knows.

What I'm arguing is that while at one point the pendulum was swinging too far one way, it's now too far in the other, and it's affecting software quality industry wide.

Testing foundation is great but that is not at all what the current interviewing system does. It hides foundation in trick questions that require you to have a moment of insight that fires off a light bulb that you can then use your foundation to solve.

For example why cant they just ask. Ok givin an array find all the pairs of the array. Great now whats the big O of that. Great...

Instead they ask some trick question that disguises the fact that you should solve this problem by finding all pairs etc.

I totally agree. My point wasn't that the interviewing system was great. Just that the problem with it isn't what it's trying to test, but how.

I have issues with people who dismiss the problem space entirely. "I don't need to know CS fundamentals to do my job". To which I say: you do. You're just doing a subpar job because -everyone else- is and that's the standard even though it's suboptimal.

(Note I don't have a CS background myself, and thought this for a long time, until I ended up working in environments that were CS heavy and seeing how "hard problems" were trivialized, so we could spend time solving "real" problems instead)

I thought about this, I thought about this a lot and I still do. Thing is, this attitude can quickly get you tumbling down the rabbit hole.

So say I'm a Java developer. But surely in order to be a good Java developer I should know the innards of the JVM, right? To do that I now have to also have to know C++ because that's what JVM is implemented in or whatever. But surely I can't know C++ well if I don't know C, right? But then I can't really grasp C well enough if I don't understand how assembly works, right? What about CPU instructions, different architectures, etc?

My opinion is that nobody should even try to drink what seems to be an ever expanding ocean of knowledge in IT. Let the people (in academia or wherever) who work on algorithms work on algorithms, let the people who write frameworks continue writing frameworks and let the rest of us code monkeys to use all those digested tools and technologies to do what we have to do on daily basis. You don't ask a common plumber to describe Bernoulli's principle during an interview, do you?

Scott Hanselman has said a few times that two layers of abstractions is generally sufficient. I'm not sure if he stole it from someone but using your example, that means to be a great Java developer, you should have a good understanding of the JVM, and an adequate understanding of C++. Do you need C++ to be a good Java developer? No. If you've got a decade of experience in both, are you a better Java developer than the guy who learned it in school and that's all he knows? Probably, but not necessarily.

In addition I think the argument falls apart the closer down you get. Granted I am no C expert but I don't think you need to know any assembly to be a very good C developer.

When your writing C you are usually thinking about how the compiler will translate it to assembly. A lot of C's features don't really make sense without understanding the underlying machine.

You can get away without knowing assembly, but you might as well just use a higher level language at that point.

Then the issue isn't with the tests necessarily, it's with employers using a test of Skillset A to screen candidates for a job that is 80% Skillset B and 20% Skillset C. That doesn't make the tests ageist (which I suspect was included mostly as click bait), it just makes the hiring managers at these companies bad at their jobs.

Raise your hand if hiring managers being bad at screening technical candidates for highly technical roles is surprising to you.

Agree, but what is the alternative from the standpoint of the interviewer? These types of challenges are simple to state, self-contained, easy to timebox.

For the interviewer for a say intermediate-level enterprise software backend position who needs to quickly sift through 50 applicants, what is the better option by way of code challenges?

Credentialing. Seriously. Have a Bar Association for developers. When anyone can do a few weeks of Googling and claim to be a software developer, you're going to get mostly crap candidates. Offload that process to a formal body, like lawyers, doctors, and engineers do. You can hire unlicensed developers, but you do so at your own risk. Then once you're talking to someone licensed in an interview, you can focus on just getting to know them and talking about their past work.

Edit: wow, I've clearly touched a nerve. Good thing your doctor is licensed to prescribe something for it instead of a rockstar named Chad who read about MongoDB once but can basically build Uber by this point.

For the professions this is quite simple as there are standards, Doctors, Accountants etc are all trained to the standard qualifications that are often internationally recognised.

How could you do that for software engineers? There are so many programming languages, frameworks out there that are constantly evolving and companies have different demands.

I think it would be a really hard problem to generalise.

> There are so many programming languages, frameworks out there that are constantly evolving and companies have different demands.

An idea that I have thought of is to standardize on some programming language/frameworks/demands that are known to be supported for a very long time (15 years?). If you need something from this buffet, you are fine and have the advantages that this provides. If you have special demands, you are on you own - which is not a problem per se, but as all special demands this simply needs to increased risk or cost.

I would love to see some sort of standardisation and rationalisation within software engineering. It's so fragmented and ego driven. Every framework is better than the last, my programming language is better than yours. Thought is rarely given to longevity. I'd much rather work with a more dated ecosystem that I know will be around of true next 20 years than the continual paradigm shifts we endure just now.

I'm exhausted and frustrated with the field as it exists just now.

Fair point, but consider that software engineering is a MUCH younger profession than, say, accounting. Consider that "double entry book-keeping" dates back to around 1340, and accounting in general is even older than that.

We've had about 60 years to figure this stuff out, and arguably the ground is always moving beneath our feat as the nature and scope of the systems we build keeps changing.

I'm exhausted and frustrated with the field as it exists just now.

Likewise, but I'm not sure what choice we have, except to hope Ray Kurzweil is right about some of this life extension ideas. If we can survive another 300-400 years, maybe things will be better.

I'm not sure of anything, but if I had to pick one thing to bet on it would be that none of us, or our children's children['s children] will be here in 400 years.

There are many jobs like that; it's not hard to some Java shop maintaining some CRUD software for over a decade, with no signs of switching. Hell, a colleague of mine just got a decent offer to develop in Informix 4GL.

You are just adding a lot of useless bureaucratic overhead. Are you sure that the problem you are trying to solve will not just be replaced with an even worse one?

No, I'm not, but that's, to be rather frank, a stupid fucking reason not to try. :)

This is maligned and dangerous. It has been shown that credentialing hurts the marketplace and consumers. Having a credential authority would not cause there to be a skill bar, it would just require all of us to pay money to a worthless authority.

> it would just require all of us to pay money to a worthless authority.

Today these authorities are universities.

HackerRank-style test are one of the few ways into a good career for lower-class people who have the skills but can't afford the credentials.

It has been shown where? Please don't forget that inexperienced practitioners also hurt the marketplace and customers. Every data breach because someone kept a plaintext password field in MySQL hurts the customers. Every lost hour of productivity because of shoddy code hurts customers.

It exists: https://ncees.org/engineering/pe/software/

No one cares though.

Hence our dilemma.

I would love to sit for a PE exam, except my state requires you to apply for permission, and one of the requirements is 4 years of professional experience working under another PE, after you're already approved as an Engineer-in-Training, which itself required 8 years of experience if you don't have an engineering degree. 15 people sat for the software PE exam last year nationwide compared to nearly 1,000 for some other disciplines.

With those kind of numbers how is anyone with the 12+ years of experience required who doesn't personally know a software PE (of which it seems there are maybe 100 in the United States based on the age of the test and pass rates) supposed to get the certification? I totally accept that my Poli Sci degree means I'll need more professional experience that a Comp Sci or Comp Eng graduate, however the requirement to work under another PE eliminates the vast majority of potential applicants.

Big companies already spend a lot of time and money interviewing candidates to figure out if they should be hired. Credentialing will turn that into either a standardized test (which is known to be inferior) or simply require a degree (which will exclude too many good candidates).

Under a hypothetical licensure system, companies would be more than welcome to hide rockstar Chad from the local CS department. They would just, for example, be more liable for damages or something. We can think of all types of licensure schemes, with all types of consequences. What makes you think software engineer credentialing will automatically work out poorly? Moreover, have you also considered its benefits? Oftentimes the world exists in shades of grey, where a system has good parts as well as bad parts. Lawyer credentialing, for example, isn't perfect, but it's better than nothing at all.

Lawyer credentialing exists because individual clients are unable to vet lawyers otherwise. Same with doctors. In both cases, the consequences of getting a quack doctor or lawyer can be disastrous... death or imprisonment, for example. General contractors can cause quite a bit of damage to your home so they often have to be licensed and bonded, electricians can cause fires so they're licensed, and bad plumbers can do plenty of damage to your house too. All of these people are likely to be hired by private individuals.

A bad programmer could rack you up AWS charges, or leak your clients' credit card numbers to an attacker, or other things like that, but these are just financial losses for businesses that don't do their due diligence (nobody's dying, going to jail, or having their house burn down).

> Moreover, have you also considered its benefits?

I don't think the benefits have been adequately explained to me. Maybe I could consider them if you told me what they are.

Part of the problem is that the discipline is very wide, and there's different mixes of skills needed:

    +-----------+
    |\ business | - excel macros
    | \     +   |
    |  \  users | - web dev
    |   \       |
    |    \      | - back end dev
    |     \     |
    |      \    | - framework dev
    |       \   |
    |        \  | - tooling
    |         \ |
    | coding   \| - OS / compiler / database dev
    +-----------+

There's no simple cutoff that can mark someone out simply as a developer. And the diagram above doesn't reflect the fact that within each slice of the coding wedge, there's a different mix of technologies, different rates of technology turnover, requiring a different mix of e.g. aptitude to pick up new technologies quickly, vs depth of experience in existing technologies, quite apart from what the specific technologies are.

Conjoined triangles of success!

There is powerful opposition in our line of work to this type of thing, particularly the standardized tests. I don't fully understand it because I think that certification exams, etc. have clear, if limited value.

I think it stems from an intrinsic starting point among many developers that programming is a true meritocracy and that all hiring questions can be settled with a judicious study of what the person is truly capable of (eg. the fabled personal github account filled with side projects). And that anything other than "hacking on the code" is a waste of time.

Of course this isn't how the real world works. In fact almost no one has the time, energy, or inclination to do impressive side projects. Gauging skills on the basis on resumes and very short interviews is quite hard and time-consuming. Hiring decisions are made on the basis of essentially social interactions and personal references.

I do think there is room for an informal credentialing process in principle, but there are longstanding mental blocks that will need to be skirted. There will need to be leadership from the top to make this happen, ie. the best, most-respected programmers will need to take it seriously first.

Another solution is law. Establish a formal licensure body and then slowly make it necessary for things like government contracts, or even private contracts.

I don't know from where it stems for the majority of opponents. For me, it stems from a complete lack of faith that the system would be well designed and even relatively free from corruption, and therefore that it would not make things much worse, while not actually solving the problem it's supposed to solve.

Certification exams already exist, and employers are free to require them to avoid the "crap candidates". Other problems, like security flaws affecting the users, can be better solved by imposing real penalties on companies.

doctors

Because the healthcare job market is such a success! Remember those accusations against tech companies for colluding to depress wages? Now imagine the same, but with State immunity!

The anti-trust class-action lawsuit Jung v. AAMC alleged collusion to prevent American trainee doctors from negotiating for better working conditions. The working conditions of medical residents often involved 80- to 100-hour workweeks. The suit had some early success, but failed when the U.S. Congress enacted a statute exempting matching programs from federal anti-trust laws.

I never said "let's make software licensure exactly like healthcare licensure". There are parts of healthcare licensure we could also criticize, but that's not my point. If you'd like to discuss specific parts of softwares licensure, I'm more than willing, but don't engage in faulty reasoning.

You don't get to decide how it'll shape up; public choice and markets don't work like that. The decision to mandate licensure unleashes forces that very often lead to the kinds of problems that I described above.

Unless you have a very good reason why it won't in this case, you're just playing with fire.

> For the interviewer for a say intermediate-level enterprise software backend position who needs to quickly sift through 50 applicants, what is the better option by way of code challenges?

Patrick McKenzie (patio11) created a lecture about this topic:

> http://businessofsoftware.org/2016/07/hiring-at-scale-patric...

If you are short of time, here is a summary:

> https://42hire.com/hiring-at-scale-c99f0665b893

> For the interviewer for a say intermediate-level enterprise software backend position who needs to quickly sift through 50 applicants, what is the better option by way of code challenges?

Code challenges are a dead end. You can have someone who passes every code challenge imaginable, but is absolutely awful to work with. Software development is more about communication and problem solving than anything else, so focus on those things first. Technical challenges are easier to overcome than personal ones! Here's my strategy:

- Resume smell test to filter out 90% of the applicants (spelling errors, run on sentences, poor formatting, poor experience, etc. If the applicant doesn't care or know enough to fix these on their resume, chances are they will do the same to your code base)

- 20min In person or on-camera interview smell test to filter out another 5% (punctuality, check for communication and personality problems, smoke test of technical knowledge)

- 1 hour+ in person interview to pick your candidate (deep dives into past projects, talk through or whiteboard a common problem in your industry, meet'n'greet with other team members)

- 3 month probation period to fully vet the new candidate and build trust.

There's no way that, out of 50 applicants, all of them are going to be equal, don't treat them that way. You need to fast track ones that are very promising and get them to meet with the team asap.

Take some Real World™ problems that are applicable to the positions are create your own. I don't mean come up with something completely from scratch; literally comb through your code base to see what kinds of design patterns and algorithms you're using. Then generalize it.

It could be as simple as: Given a list L of accounts, return only accounts from L that match key K. That's programming 101 type stuff that is applicable to the job.

Or it could be more abstract and open ended: Given some user-submitted form data, what steps would you take to move the data from the browser's front-end to the database. Assume the database has been created. Tailor the question to the technologies on used on the job (Spring, .NET). This type of question gives you a whole host of information about the candidate such as their overall knowledge of the framework, their areas of expertise, potential weaknesses, as well as the ability to nudge the interviewee in the correct direction if they're stuck.

Interview questions don't have to be clever. They don't need to have obscure edge cases to trip people up. I firmly believe that interviews should be about gauging candidate ability to be functional on the job rather than solve the riddle (how many piano tuners are in Chicago?) or non-applicable CS (create a script/program that can output the nth row of Pascal's Triangle [ignoring int overflow]).

I recently had a round of interviews for a senior engineering position.

Luckily it was on hackerrank, but only as a way to store the exercise instructions with a timer.

The test itself was not terribly difficult, build a small mobile app (I am a mobile dev). It was still enough to demonstrate how you can architecture an app, good design patterns, clean code, etc

After that I had another round of interview with that same company.

I feared that ridiculous whiteboard was on the menu. One of the engineers reassured me when I inquired about the content of this interview : "we think that wb are bullshit, we just want to discuss".

In the end I was offered the job, with a really nice compensation package. I should mention that I live on another continent and did all these interviews remotely.

Is this a general trend in the silicon valley area ? or was I just lucky to stumble on companies that don't believe in whiteboard questions ?

I get idea that these questions are exactly same from that you get from clients and incompetent bosses, where you expected by other side to fill the missing bits and show your brilliant intuition to solve the right problem.

they are entertaining to play with(solve), though i haven't encountered any serious algo problems during my line of work in 15 years. my friend, game coder, always does.

Reply on news.ycombinator.com