I got asked LeetCode questions for a dev-ops systems engineering job today
reddit.com
reddit.com
The code test is one of the more annoying facets of interviews. On the hiring side, it's an efficient way to screen and vet candidates for basic skills. It's relatively low cost to the employer to have a recruiter issue the test.
If you come up with your own in-house code test, it still requires time to evaluate and grade the submissions. There's also a fairly large cost to creating an in-house code test (I know from experience). You have to balance complexity so that it requires enough skill to pass, but isn't so onerous that it puts off too many candidates.
Using something like LeetCode removes all the cost from the employer side. Someone else maintains it for you, and you get a canned score at the end. It also depersonalizes it, and provides an opportunity for people to game the system by memorization. The same applies to using canned whiteboard/algorithm questions.
I think most of this is a colossal waste of time. It turns away people who don't want to spend hours (or days) on each interview. Imagine applying for even a handful of jobs where each one expects you to do a take-home test that can take a full day. Who has time for that? There's the "if they really want to work for us, they'll take the time to do it" argument, but let's face it, most people are not that passionate about where they work until they actually work there.
As a candidate, I am loath to spend my time on these. As someone who's been doing this for 30 years, I'm also a bit insulted by having to do the monkey dance over and over again. At this point in my career, I don't bother applying to places like this. My resume speaks for itself.
Of course it makes sense to vet the technical skills of candidates, but you can learn a lot more from 15 minutes of questions than you can from a code test. If you present a candidate with some abstract questions and a couple of targeted detailed technical questions, you can tell if they've got the right problem-solving mind and if they've had experience with the technology you're working with. It's also a much more personal experience.
I agree with you in principle for a programming job, but from the post, this was a dev ops job, and the candidate was asked a dynamic programming question. There was no link to the JD, but I suspect it had little by the way requiring a CS degree.
I have no way to prove to veracity of my story, so feel free to add it to the "urban myth" pile. But it was extremely bizarre to experience.
In over 25 years of interviewing candidates, this approach has never failed me. I've never asked for whiteboard coding. Whiteboarding architectural concepts, sure.
The only caveat is the interviewer needs to have substantial experience in the same field to spot the wheat from the chaff in a casual conversation. If the interviewer is very junior, I can see how this might not work. Solution: don't have your most junior people run the interviews.
I agree asking highly specialized, deep CS-algorithm type problems is usually a terrible indicator. I typically start with a very rudimentary question that could go off in various directions and to various depths as time/progress allows. I let candidates choose the programming language, and to use either whiteboard, pen & paper, or laptop with choice of IDE.
I've seen candidates who claim to have years of "senior" experience struggle to get the basic syntax of their "language of choice" right - to the point where a competent hobbyist who spent a weekend doing a couple tutorials would wipe the floor with them.
There's just something about coding that makes it impossible to truly evaluate a candidate (for a developer role) unless you see them actually code.
It makes zero sense that someone can have a deep technical discussion but they're not able to write fizz buzz (which by the way is a crap exercise). Sure, if one is asking general questions about agile, TDD, SOLID and other fluff, anyone can answer those.
Yeah it happens, but not as often as you think.
The complex questions, if I can get answers are awesome. They are also the ones that are hard to objectively describe. Based on the number of blogs read, candidate charisma, my own work pressures - there is a big chance a senior enough person can wave their hands through those. Hence the tilt towards the surety of coding.
Also TBF, in my 1000s of interviews, we have always had a couple of rounds (Bar raisers / System Design / Architecture ... whatever) which try to get the fuzzy input. We get a sense, but that's not in depth as you might imagine. OTOH we have a clear cut answer in the problem solving world that gets us a bit more closer to the hire/nohire decision.
For instance, currently my title is “senior software engineer”. But I am the “Directly Responsible Individual” for a project that one of our clients is paying for. It will be added to our core product.
This means that I’m expected to interact with the client and BA to come up with the SOW, work with the product manager/UX person to design the interface, do some of the web work, write the backend using one of our approved languages (C#, Javascript, Python) , design the database schema, do all of the CI/CD setup and to know the AWS fiddly bits and know which services are appropriate for the asynchronous processing requirements.
I’m no special snowflake, this is life in small companies since my first job 20 years ago.
I don’t interview candidates for algorithms. I interview them to know whether they can hit the ground running on as much of our (industry standard) technology stack as possible, will they make good architectural decisions, can they ship a feature across the finish line, how fast they pick up new frameworks and will they embarrass us with customers and senior management.
LeetCode ability does us no good. I want to know whether you can translate real world business requirements into a working system - including knowing which questions to ask when customers/product managers come to you with “XY Problems”.
So yes - we have to do all those things. I am directly responsible for ALL of data analytics infrastructure in snapchat. Pretty much most senior engineers have a similar scope.
We might be talking over one another though: I am not saying any of that is useless: I am saying they're hard to measure. We still do it though. We also do the coding round as something that gives a more definitive datapoint.
It’s really not that hard to ask a few technical questions based on what they purport to know, the soft skill behavioral questions, some design questions, and ask them about previous projects. The hands on part is the part where they fix the code to make the unit tests pass.
I wasn’t asked a single technical coding question for my current job. I was given real world questions about issues they are having with process and how I would go about (re)architecting some processes.
Now that I think about, my job before that when I was interviewing for the dev lead position I was only asked about process, system design, and a lot of high level theory - both jobs were hands on coding positions. Both of the managers were highly technical with current programming experience.
But, I’m not completely disagreeing with you, if you are solving hard problems(tm) at scale you need people who understand algorithms and you may not be able to use off the shelf solutions - I would say Snapchat qualifies for “at scale”.
But, for you typical yet another software as a service CRUD app or your typical dark matter corporate developer - and those positions are a lot more numerous than HN seems to realize - the complications aren’t in the complexity of the code, it’s in translating the business needs to code and shipping software that the company needs.
As strange as it sounds, the “coding” side of software engineering has gotten a lot simpler 20 years into my career than it was when I started (C, C++, and a little x86 assembly). The “engineering” side has gotten a lot more complex and requires me to know a lot more technology.
I don’t think it’s because I am better “coder” now either. When I got my first job, I had already been a hobbyist for a decade doing assembly.
I have been able to weed out candidates by asking them to do an isOdd() function as part of a pair programming exercise. It was still the Bozo filter before I gave them harder problems. Similar in complexity. But even that, I’ve done with an IDE with a skeletal class and failing corresponding unit tests.
Something as simple as having somebody write a little web application that takes a request, calls an external service, parses the response, and returns a transformed response, filters out the bozos wonderfully.
Some of them couldn't even get nested for loops to work correctly.
If you're hiring someone to write code for you, you need to have them write code in the interview. There's no substitute for it.
Obviously, you can improve the experience - let them bring their own laptop and code on it, no problem.
Similar to how not everyone is able to perform well while having sex-on-demand in front of a camera -- these same people are more than likely able to have sex when not under pressure and observation, and probably are quite adequate while doing it.
My personal experience in software development spans decades, I'm quite competent and I have definitely "choked" at the whiteboard, just like you're describing -- for the same reasons I don't do well speaking in public, I'm not comfortable doing it, and I'm just not cut-out for it... I don't think it's a good measure of actual problem solving and/or software development competency.
That's.... a much more accurate term than the "rock star" that gets tossed around.
Frankly I think most people suck at being interviewed initially, but you can definitely improve with practice. The same as public speaking and interviewing candidates.
Thing is, I still had a good 3-4 years of professional experience with stellar reviews, some hackathon wins, on top of having coded throughout my entire life prior. But when I was up on that whiteboard, I pretty much froze for that entire segment. Whiteboards are just a different skill and if you don't regularly code on them at work, you better buy one and practice at home because using an editor does _not_ translate to coding on a whiteboard.
I'm glad you mention this: "Obviously, you can improve the experience - let them bring their own laptop and code on it, no problem", because I did better in interviews that followed which allowed for coding in my usual working environment. Heck, I even passed a technical screen before this company brought me on-site where I live-coded some elevator problem they asked me to implement. After bombing the whiteboard, I wouldn't blame them if they thought I had paid someone else to solve the tech screen.
Also there's no point in bringing someone in for an interview before they've shown any coding experience. It's a colossal waste of time for everyone, including the candidate
We give an invite for a phone screen. It's just a chat so both candidate and interviewer can make sure that the job on offer and experience of the candidate match up. It's also a nice time for candidates to get a decent idea of what our company does so they can bow out if it's not really what they're looking for, which is good for everyone.
Then we send a tech test (take home). It's a simple challenge, with a lot of freedom. We give a guide time of ~90 minutes, but also let the candidates know that they can do more or less if they want. We ask the candidates to submit a written response as well, so they can explain if they're happy/unhappy with the test and their submission. Didn't have time to refactor as you wanted? Didn't write as many tests as you would usually do? That's fine, just let us know generally what you'd have done so we can take it into account when looking at it.
Then it's onto the face-to-face interviews etc.
The tech test segment is really enlightening. A lot of the time we can figure out a lot about the candidates ability to create software systems and communication skills just from the test. Most of the time, we spend a non-inconsequential amount of time looking at the test submissions. If a candidate was good enough to write it, we should take the time to really see what they did and review it. Sometimes we get really poor tests back, even from candidates we were enthusiastic about. Other times we get pleasantly surprised, with candidates who phone screened poorly smashing the test and giving a wonderful submission. I think it's immensely important to see what a candidate does with actually writing code. I also think it's important to make sure we see that prior to inviting them in, as they need to take time off of work, get to our offices and prepare for it. And even when we really want to progress with a candidate, the tech test gives us something to talk about to get to know the candidate better, and hopefully have a conversation that allows them to ask us questions as well.
It's as if you asked an auto repair mechanic about designing involute tooth profiles on fishbone pattern gears in a transmission. Sure, if they could do that, then yes they'll be a great technician, but it has nothing to do with brake jobs and oil changes.
EDIT: Per the responses in the reddit thread, the question was: "So basically it was a variation of the exact change problem (I realized afterwards). You had to validate a string of size n. 1 chunk of 8 characters could have 1 format, 1 chunk of 16 characters another, and 1 chunk of 24 characters another. You essentially had to determine if the string was valid. So in other words, find a combination of these sizes in which the string could be validated. I had 30 minutes."
So, whichever side of dev-ops you lean to, there’s a reasonable answer that should be within your field of expertise. And given Palantir’s line of business (the company involved), I wouldn’t be surprised if dissecting a byte string with limited information about it’s structure comes up from time to time in their actual work.
When I am at a whiteboard I am drawing very crude architectural diagrams.
(sorting an array, with any algorithm you want)
On the other hand, would you accept a bubble sort as a solution? Or the most degenerate case- a bogosort ;) ?
Accepting things like bubble sort also makes the exercise doubly pointless.
Now if someone did a BogoSort, I would hire them just for their ingenuity and sense of humor.
I like the people I work with to have a good sense of humor.
And if you are unable to sort an array it's a strong signal that command in all other areas of CS is lacking. Does C++ has a standard library for RSTP? Do you think a person who can't sort an array after 5 years of school would be able to fix a bug in a switch' spanning tree implementation?
Does C++ has a standard library for RSTP?
Maybe not, but I know that there are very few new problems under the sun that most people face and someone has solved the problem before. I also know how to use Google.
https://stackoverflow.com/questions/1460010/best-c-rtp-rtsp-...
(before you go on googling again, the algorithm operates over the switch port forwarding fabric, you can't really use a library)
Both approaches however would be beyond the skills of a programmer who can't sort an array. And really it's not a high bar, I don't understand this obsession with anti-intellecualism. Everyone likes plumbing analogies, well I wouldn't trust a plumber who doesn't know that water generally tends to flow downwards - it's this level of CS knowledge we are talking about here.
In your case, at the end you've only figured out that your candidate knows how to sort an array, which even high-school students can do. You can confidently say that your candidate has a level somewhere between student and Donald Knuth, a pretty useless piece of information.
Is there any connection between sorting arrays and spanning trees? If I'm not mistaken, the latter involves graphs.
So I'm not quite sure what is your beef with it exactly. Do you think I should have used a more complicated task? Like, Dijkstras' on a graph? Are you sure that you wouldn't find that objectionable too, given that sorting an array is apparently asking too much?
But, you are hiring for a software developer, who should be able to translate business problems that you face everyday into a working system. Don’t you have any real world scenarios that you could use to test on?
a) is too easy
b) doesn't have enough meat to differentiate between junior / regular / senior
c) covers very little if anything at all from what a SWE does on their day to day. OTOH I guess/hope you had more things, not just this.
This was the only coding test but we also either talked through opensource code candidates brought in or discussed some parts of one our real project here.
Who the heck writes code on a whiteboard for production anyways?
There will always be unknown unknowns. We are just trying to cover the known variations and if he has any unknown knowns[1].
At the time of this writing, I am working with someone who has 10+ years of experience, including former positions as a lead developer. I was involved in the interview process and made the recommendation to hire, as they were able to answer a lot of domain-specific questions very well and had a good knowledge of the ecosystem. Fast-forward three months, and I am spending hours per day sending changes back on code reviews for features that would have taken me 30 minutes to implement. Every review has at least one conditional that always evaluates to true. Yesterday I found a try/catch block that always throws inside of the try block. The crown jewel was the following for-loop (psuedocode):
for i=0; i<=3; i++:
try:
doSomething();
catch:
if i<3:
continue;
else if i==3:
break;
And the best part is because of political reasons and difficulty hiring a replacement, we can't fire this person. I now include coding exercises in my interviews. I typically focus on reviewing code with bugs and refactoring bad code instead of forcing them to whiteboard.1) Does this code accomplish the goal? 2) How would you improve this code to make it more readable? More efficient?
Personally, I think I'd rather work with people that know how to review code and provide feedback/commentary, since that usually means they can write good code themselves. It should lead to a better team dynamic.
There are plenty of programmers who would have jumped right in and explained what the code was doing (surface appearance) and missed the simple fact that there was nowhere enough information present to actually know what that code was actually doing.
So even with you two questions, which are a start, getting them to indicate they understand the context/environmental problems can be a more challenging process.
Edit: for example, when interviewing for a react/redux position, one of the first questions I ask is "if you need to make a network request in an app using redux, where do you put it?" at least 3/4 interviewees answer "in the reducer". From page 3 of the "Basic Tutorial" in the redux documentation:
Things you should never do inside a reducer:
Mutate its arguments;
Perform side effects like API calls and routing transitions;
Call non-pure functions, e.g. Date.now() or Math.random()....at the salary you are offering.
I see many people hiring adjusting for cost of living but not adjusting for the place being less desirable to live. Also, given the reputation of FAANG and similar, working for them is viewed as something of a future investment. That is a benefit your company likely cannot offer. So even if everything else was on par with a FAANG in terms of pay, benefits, and working conditions, ideally you would have to pay a slight bit more to make up for the difference. Granted, the market is never that rationally acting in any individual case.
Having seen numerous IT managers having trouble hiring, the most common shared issue is too little pay.
If you're only sourcing the same 75% of active job seekers who are bombing most of their other interviews [thus leaving them available to come in and bomb yours as well], start advertising your positions elsewhere or trying to recruit passive job seekers/encourage employees to scour their own networks.
If you don't change something, you're likely going to keep getting more of the same.
Which is what was said three posts up.
Edit: I don't think you are understanding what I'm saying. We are offering higher than average pay for a job with average requirements, and we haven't been able to find average candidates. And I'm well aware of average, upper, and lower bounds to salaries here.
“We can’t afford that!” “But you can waste money on two crappy devs?!”
(If you only have budget for one, the cash flow doesn’t work, of course.)
Yes, probably. Why do you need a supply of new front end developers? Is your workplace exceedingly lethal?
Have you considered that paying $2XX/year is a signal? It attracts a certain type of candidate and you can't extrapolate the talent level of the entire market just based on your experience with those that would try to land a $2XX/year job. Where I live, trading firms can pay that much and more, with potential bonuses that would make your eyes pop out of your head. Yet many extremely talented people I know would never submit a resume: "No thanks I like having a life." Many people I know that used to work in that environment think of it as an insurance policy: "at least I know that if I ever need money I can go back."
> We are offering higher than average pay for a job with average requirements
A lot of people will pass you by and assume that something is really wrong at your company to need to offer so much money for an average job.
At least one of those two pools exists, because if not it means there are no qualified developers looking for a job, in which case perhaps the issue is with the definition of qualified.
Also, there is the possibility they are getting great applicants that are filtered out before it reaches the individual currently here. It may be possible HR or some automated system is filtering them in such a fashion that good candidates are lost. Maybe HR is demanding 7 years experience in a 3 year old technology.
What we can reasonably assume is that the hiring manager is not finding qualified talent and qualified talent does exist. The idea that qualified candidates are only applying to FAANG or unicorns seems far less likely than something about the position is the reason they aren't getting anyone qualified.
Sounds like it'd be more effective than spinning wheels without traction. :/
Edit: I mean this as in I know several people including myself who apply for 2xxk/year and remote. Post or dm me and I'll see what people I could send your way
> Honestly you don't seem to be a well put together environment
That is a pretty bold and unfriendly statement to make based on very little information.
> you expect to hire experts in some random framework
We're hiring front-end developers to work on a react and redux application, which is kind of a de-facto standard for single page applications these days. We have an app with a highly dynamic UI that requires immediate user feedback and must work offline. I don't think we're misapplying the technology.
> I'm not surprised you're only getting code monkeys who have used your framework in passing
Again, we're interviewing people with several years of professional experience using react and redux listed on their resumes. We're also not tailoring the interview process to only react and redux specific questions, but if someone claims to have been paid to write react apps for three years, I'm going to ask them some questions about writing react apps.
I believe you'd have much better results if you ascertained the applicants technical ability before the interview, and instead spent the interview getting to know the person, to see if (s)he had a positive and friendly personality. If none of your applicants have side-projects or code on GitHub then perhaps you're searching the wrong places, and/or failing to properly promote the position (especially if some of your developers are earning 2xxK). I'm sure if you posted the position in HN's 'Who's Hiring' and included a 100+k salary, then you'd get flooded by talented developers that have open source code/impressive side-projects. And if somebody with an impressive resume doesn't have that, then you can just request a small take-home assignment that's related to the work you do. If you'd like to avoid getting flooded with resumes', then you could also just reach out to the developers you find interesting on GitHub.. considering your salary range then I imagine you'd have a very high interest rate.
I work with lots of really, really good developers who struggle a bit with JS. Most of it is because they come from a totally different background, and the other part is Javascript in general just makes things difficult.
My experience with Haskell is that it's the most expressive and flexible environment for exploring different ways of structuring code that I've ever used. I've seen everything from extremely-imperative code carefully managing memory allocations and munging bits, to very abstract high-level libraries that declare abstract computations that can be interpreted or executed in a variety of ways.
I'm very confused and surprised to see someone assert that Haskell forces a particular structure on your code.
(This is what I love about Haskell, BTW, so please don't read this as a criticism.)
This isn’t a criticism of Haskell: there are plenty of benefits to forcing declarative IO and limiting the language (roughly) to features that don’t break type-inference, but those advantages of Haskell make it much less enjoyable to experiment with programming paradigms in.
I knew almost zero about web dev (I had been a decent C++/Java/Scala backend programmer) when I was hired to work in a web dev role -- I spent the first two weeks doing RoR tutorials on the internet and was just as productive as anyone else a couple of months after that.
Sometimes (many times) if you can get someone who is more of a generalist and is capable of diving in to gain the necessary skills is far more valuable. They are quite often the first to be discarded from the interview process.
Many of the psoitions I have participated in required handling new tasks that were not within the broad scope of what the position was expected to handle. Yet, having the attitude of "let us see what is needed", meant that I was bale to be functionally more effective in that position, including being the "go to" person for those who needed problems to be solved that their normal IT groups would not handle.
I have found too many "code guru's" to not be capable of thinking outside of the box whaen it is necessary. You don't have to be a "brilliant coder" but more of a person who can develop an understanding of the "problem space" of those having the problems that require solutions.
Perhaps you shouldn't be using complicated faddish framework then?
> interviewing for a react/redux position one of the first questions I ask is [very vendor-specific technology question]
FAANGs don't hire people for "react/redux positions", they trust smart, conscientious people to learn it when needed. Maybe that's your problem.
Maybe the good people aren't already "react/redux programmers".
Your hiring woes will get easier to handle once you start treating the Competent Sr React+Redux Developer role as rare. React is certainly popular tech, but solid knowledge of the ecosystem is still far from common. All the places I've done react at had ~10 frontend capable devs, but only 1 or 2 people who really knew what was going on.
It’s a beautiful system that brings a lot of the benefits of a Free-monad style effect system.
Bless you sir, you are keeping the code legible and clean for everyone else on your team and doing a great service for your company — even if they don’t appreciate it.
The standards for what a company wants from their programmers can vary wildly depending on the location. This speaks to a broader problem of not having consistency across the industry and various locales.
It's really a lack of training provided by companies and that most 4 year degrees don't actually prepare people to write good maintainable code so new developers have to luck into finding a mentor that can teach them these things.
When I said 'the midwest' in my original post I didn't mean places like Chicago, St.Louis, or Indianapolis that pay industry standards adjusted for cost-of-living. It isn't out of the realm of possibility though for someone to get 10+ years experience in a place like Lexington and have them still not know basic things EVEN after having been trained.
Again this is just my anecdotal experience, I don't mean to insult anyone from these places. This is about brain-drain and salaries, not about anything intrinsic about a place or their people.
We all know small businesses don't care about code quality. Ditto that best practices take a long time to trickle down from ivory towers to business, academia, to bootcamps.
Companies pushing the status quo tend to cluster. Poster probably shouldn't have picked on the midwest, it could be anywhere outside "cutting-edge" software centers.
The "Joel Test" was a thing, for a reason.
While I agree a coding exercise is valuable - this is the actual problem. The way I test for that humility is by asking "tell me about a time you personally did a poor job." I then follow up by asking for a time they identified a failure of a colleague. I look to see if they're empathetical, open to being wrong, and declaring they're always trying to improve.
People get step-by-step strategy for this kind of question, what the interviewer is looking for, and what notes to hit. And sometimes prep them for the exact question, so that they can have a prefab answer ready.
Then we might be selecting for more like what used to be MBA students, or at least people who know how to play along, and say the right corporate going-through-the-motions things.
Some organizations will want to select for test-prep specifically, as a good predictor for the kind of employee they want, but I like to think there's other organizations that decidedly don't want that.
Folks are given all sorts of interview advice about leading to claim things like “my greatest weakness is that I work too hard”.
If I ever hire an engineer for my business -- I hope it'd be someone I've already worked with and know -- but if it had to be someone random, I'd pursue this method:
1) Solicit Resumes
2) Pick N candidates
3) Phone screen to make sure they aren't complete assholes
4) Stack rank & pop top candidate
5) Pay them to develop a feature on contract
6) If satisfied, repeat step 5 until ready to hire; else repeat step 4.
I think it'd be important from the start to let candidates know they might need to do a week or more of work (for pay) before getting a final hire. Some people would be put off by this, sure.
But I suspect the people put-off would be people desperately seeking full-time employment. I think it might attract a lot of talented people that don't want to go through a 4-week long ceremony of an interview process that takes up days of time with no pay, though.
I feel this filter will only leave behind those who are currently contractors that are interested in maybe going full time, and the unemployed/employed who are desperate enough to go along with such a scheme. Foregoing the typical crappy interview process is one thing, but working, even for money, without any certainty as the end outcome would certainly turn me off.
To be honest, this setup seems like something that would be hatched by a scummy company to constantly string you along with a carrot that will never come. I am not implying that you are that type, but there are enough horror stories out there that I would avoid ever getting myself put into that position.
Must be looking to get out for some reason. Doing a small side-project shouldn't be a big impediment.
> I would avoid ever getting myself put into that position.
Extreme risk-averse behavior can cause problems on the other side. Easy enough to give a small-project a try and cut out early when red-flags are actually encountered.
Just the cost of doing business, IMHO.
If you are doing well at a company, I am not sure how you make this seem like a good value proposition, especially in this tech job market. Another way to put it, with a personal spin- I don't work at an actual FANG, but a company that arguably pays better, and their would have to be a very large upside at the other end of this process to make it worth going through. Things would have to be very bad for me to give me up my deep in the six figure job to spend a month interviewing at some place unless it had some legendary track record.
This is all theoretical though and not really worth arguing over. I'd like to see OP try this out and report back the results, good or bad. I could be wrong, but I would be willing to wager that you get a lot more resumes just out of a bootcamp as opposed to people with proven track records currently at prestigious companies.
Folks do side projects all the time. How often do they describe it as a "huge risk?"
“Man”-up, and fire fte early if needed.
Desperate people won’t be put off by this. If they’re desperate then they’ll have time to do your contract and the money will be attractive. They can keep job hunting in the meantime so there’s no downside.
That’s not to say this can’t produce someone good. I got my first full time programming job like this and I like to think I’m not completely useless.
And alas, I don’t have any answers for how to do it better....
For example, I like this one and I'm gainfully employed. Much rather work on a real side project than useless, unpaid coding challenges any day. For me, the first has a 98% chance of success, while the second has a 2% chance.
How do you mean "ready to hire"? You're interviewing and giving audition gigs before there's an actual job opening?
It will be very rare to find a good candidate willing to quit a well-paying job to go on contract for you.
This is a completely different (and actually relevant) thing compared to leetcode/hackerrank riddles.
If it does, you may consider to improve the quality of your tests.
If you don't have all of those it is your fault since you are cowboy coding instead of doing engineering and at this point he is playing along.
Anyhow, if it get the job done, just carry on.
The importance of a random snippet of code of a random company is so small that is pointless to discuss about it.
Just don't let him do any design work and be quick to throw away bad code and refactor.
You got all your tests and procedure in place so it should not be that hard.
We are really overestimating the importance and difficulties of our work.
If somebody code is routinely denied to land on code base because of some of the issues catch by automatic tools, it is reasonable to assume that the quality of the code and the thought given to the code will improve.
Is this person routinely writing code so bad that passes all the automatic tests?
By the way, such code ends up having a quite high cyclomatic complexity.
Yes, a linter or a test cannot know if you are writing silly, unnecessary code, or re-implementing functionality that already exists elsewhere, or making multiple changes downstream when you could make it once upstream. Only a human can see those things, which is why we have code reviews. And for posterity, yes we are using CI, pre-commit linters, unit tests, integration tests, end-to-end tests, and have formal branching, merging, review process, and coding standards. There are no cowboys here, and nothing that you are suggesting will prevent someone from writing low quality code that requires an excessive amount of time to protect the codebase from.
Believe that's the root problem, right there. Gun-to-head coding problems aren't the solution.
the try always throws an exception, and if i<3 resumes the code. But on the last iteration through it breaks so will not execute code after the exception is thrown. The first 3 times it will execute code after the exception is thrown. Does continue return to the beginning of the for loop or resume execution where the exception was thrown?
If continue returns to the beginning of the for loop then I could see that the code is pointless. But if continue resumes execution inside of dosomething the line after the exception was thrown, then it could actually be useful.
Is your issue that the code does actually work, but conceptually it is horrendous to implement the branching in that way, or is it that the code doesnt work at all, or is that it actually does nothing?
When I see things like this I integrate them into our interview process.
Scenarios like this scare the bejubus outta me, it's like a business logic landmine waiting for a refactor tractor to drive over
Edit: I apologize - I wrote that comment before reading the rest of the discussion, after which I had more context.
This is a very hard thing to really make a part of your thinking process, in particular because nobody wants to come to terms with the fact that they might be in the latter group.
That said, this does seem like another reiteration of the "senior engineer couldn't write fizz buzz" story, though this particular example is amusing.
I can write fizz buzz. Hell, I've interviewed enough that I can print all permutations of a string (or all permutations of a set and/or subsets of that set). I used to be able to implement merge sort in 45 minutes at the whiteboard, but I can't now, and I never want to be able to do this at 45 minutes at the whiteboard again. I could certainly detect a cycle in a linked list, and I could do DFS and BFS on a binary tree. I might be able to reason through how to find the last common ancestor of two nodes, or find all matching subtrees in a binary tree if I could hack around for a while, but at the whiteboard, in 45 min? Nah. So, no-hire.
It's time to stop pretending these tests are about weeding out people who lack very basic coding skills. Tech interviews are one of the reasons I'm not enthusiastic about recommending a career as a software developer.
But, 45 minutes for mergesort? I mean, it's not fizzbuzz, but this took me seven minutes, and it passed the test on the first run (well, not counting the test runs that verified the test was invoking an empty function and correctly reporting that None wasn't the correctly sorted list). Does it have some kind of subtle bug I'm missing?
def mergesort(items):
return (items if len(items) < 2 else
list(merge(mergesort(items[:len(items)//2]),
mergesort(items[len(items)//2:]))))
def merge(sa, sb):
ia = ib = 0
while ia < len(sa) or ib < len(sb):
if ib == len(sb) or ia < len(sa) and sa[ia] <= sb[ib]:
yield sa[ia]
ia += 1
else:
yield sb[ib]
ib += 1
def ok(a, b):
assert a == b, (a, b)
ok(mergesort([3, 5, 3, 1, 10, 11, 12, 0, 1]),
[0, 1, 1, 3, 3, 5, 10, 11, 12])
If I were doing it on a whiteboard I'd probably want to use something like http://canonical.org/~kragen/sw/dev3/paperalgo rather than Python, but I can't imagine it would take 45 minutes. Do you mean in assembly language or something? In minimal space (none of the logarithmic number of copies required by this version)?I agree that these are not very basic coding skills, but they are kind of basic coding skills, I think? Like, it's covered in the first part of an undergraduate CS curriculum, isn't it?
Unfortunately... yes to both your questions. Yes, it is certainly covered in basic undergraduate "Data Structures and Algorithms."
And yes, I would have trouble writing what you just wrote in 45 minutes at a whiteboard, unless I studied up in advance. I know the algorithm in the back of my mind, and I'd be able to hack away at it. I don't mean in assembly, or in minimal space, I just mean a reasonably efficient implementation of quick sort in a high level language like python ruby or java. Again, I could do this with a bit of study, though I'm no longer inclined to do that study. I can't think of a reason, other than interview exams, for me to front load "into to data structures" into short term memory for a midterm exam one. more. time.
Maybe it does say something about my skill and mind set as a developer, that it would take much longer for me to do this than it would for you. I'm ok with that, as long as we're clear on what "basic" coding means.
I maintain that it is an immensely different thing to say you're weeding out people who don't have basic coding skills when the test is "fizzbuzz" vs "implement merge sort".
I certainly agree that mergesort is a lot harder than fizzbuzz, more than an order of magnitude. Fizzbuzz is well under a minute.
I'm pleased that you enjoyed my comment and code! I was reluctant to make it because I feared you might interpret it as an attack.
I do something like:
for i=0; i<3; i++:
try:
doSomeIoWork()
catch:
if i<3:
# maybe some network/disk issues retry?
sleep SOME_TIME * i;
continue;
else:
# tried too many times now quit
raise
Am I doing something stupid here? for i=0; i<3; i++:
doSomething();
if i<3:
continue;
else:
break;I've had developers balk at other developer's code for writing a series of like 100 if-else's until I show them, they do it themselves only that they scatter it. You could say this thing about nearly every OO programmer out there.
Everybody writes code that mutates state in a way that produces the kind of code you mentioned as an example.
It's only that some developers think they can never write bad code, while thinking other's almost always do.
This is one reason why some shops hire only OCaml, or Haskell or Clojure devs because bad devs tend to use those languages less. So you buy into a great community by default.
catch:
pass
(That is, if the pseudocode isn't making a Rubyish distinction between exception handling and control flow [rescue vs. catch] it's just an extremely verbose way of silently swallowing exceptions, and if it is making a Rubyish distinction it's an extremely verbose way of doing nothing and the whole try/catch construct is unnecessary.)The problem is that this doesn't scale with talent and/or experience: I certainly wouldn't be better at coding interviews than I was out of college. People who aren't great at hiring take an approach that's valid at the low level and extrapolate it well beyond the bounds where it makes sense.
On the other hand, this only works if the pedigree is recognizable and reliable. I can be pretty confident that I don't need to test basic coding literacy of someone who was an engineer at Google for four years, but I've interviewed people with ten years experience and great references who were clearly dumb as a brick during the interview. In one case, we hired the guy over my objection and spent the next year fighting the fires he'd set whenever he touched anything (the founder eventually found some relatively unimportant, isolated niche for him to keep busy in).
It's a really hard problem, and most complaints I've come across are generalizing from their skewed view of what the candidate pool is like (usually assuming that most candidates are roughly as talented as they are, when the reality is that many candidate pools consist of some of the dumbest people you're likely to ever encounter)
[1] ie, practically everyone
Industrial psychology has consistently found that IQ is a better predictor of job performance than experience.
The reason being is that intelligence is defined as the ability to acquire knowledge and skills. Intelligent people literally learn faster than others. The skill acquired by a low IQ in five years is often surpassed by a high IQ person with a few months of experience.
[1]https://www.jstor.org/stable/20182140?seq=1#page_scan_tab_co...
If hiring managers are aware of this, then it might be a good idea to apply to Mensa. If you're able to get in, maybe putting "Mensa" down on your resume isn't that bad of a thing. (I've always felt it would make one come off as a douche, but maybe that's not so true.)
That doesn't mean I am necessarily on board with the fashionable view that IQ is meaningless, but I think to the extent there are people whose IQ is disconnected from their abilities/achievements, it's going to be the ones who are preoccupied with it. So I would think belonging to a high-IQ society would send the wrong message.
There's also this: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4557354/, which says "Supporters of IQ testing have been quick to point to correlations between IQ and job performance as evidence of test validity. A closer look at the data and results, however, suggests a rather murkier picture."
> The skill acquired by a low IQ in five years is often surpassed by a high IQ person with a few months of experience.
This looks wrong to me, most things take more than a few months to learn.
All it takes to be a good engineer at the lower levels is a sufficiently high IQ/creative problem-solving ability. Put someone like that under the mentorship of someone who's learned good engineering habits and (barring severe behavioral problems), you'll have a pretty great junior engineer in a few months.
Otoh,Experience and other habits and knowledge do become more relevant at higher levels, only because the responsibilities of the job shift and grow, such that you can do a lot more damage in the "learn by exploration" period of becoming good at something by just being really intelligent and learning on the job.
I currently work with a bunch of physics and math and CS PhDs, people who are likely as intelligent as I am or more, but I'm a lot more equipped for our job because I'm applying that intelligence in the context of understanding how engineering systems work, at both a technical and personal level. There are tons of engineering habits that are second nature to me that never occur to them; if they shadowed me closely and I dedicated time to explaining the reason I do every little thing I do, then they'd pick up those habits in the same way junior engineers do.
That is to say, as one would expect, the relative supremacy of IQ over other skills like conscientiousness (or yes, experience) diminishes at the higher levels. Everyone I work with would be considered "really smart" by any reasonable measure, but it's not purely the smartest who are the most effective anymore, the way it was when we were entry level.
I think people genuinely experience this but are unwilling to consider that their pool is not the random sample they expected. Something subtle is filtering out (nearly) all the non-dumb people.
Have you ever seen a personal ad with a long list of characteristics that are unwanted?
In fact one might say that the Aeronautics industry is in far better shape than the Software industry, a failure in one of their systems results in people dying but I can go to any major tech companies websites and experience a bug on any given day.
A thorough interview in physical engineering disciplines will often expect you to demonstrate/validate something about your technical skill in-person, unless you are being interviewed because people already have validated knowledge about your capabilities (e.g., hiring a known individual from a competitor that is well-respected among peers).
To your point, a physical engineering interview might be more about "spot checks" than "assume the person is lying so ask them to complete a bevy of undergraduate final exam questions". In the software world, maybe that would be akin to starting with FizzBuzz and finding other ways to get solid proof the person can implement, not just talk. To me, if you have open source work that can be read by an interviewer, I would find it disrespectful for that interviewer to act as though the work doesn't exist and they should verify from scratch you can actually write working code.
It's comparatively a lot harder to just have a conversation with a software person and be certain they have the technical competence they claim vs. other technical realms. I tend to think it is the nature of the subject, as well as the salaries drawing in more charlatans than in physical engineering realms.
> In fact one might say that the Aeronautics industry is in far better shape than the Software industry, a failure in one of their systems results in people dying but I can go to any major tech companies websites and experience a bug on any given day.
This probably has more to do with Aeronautics actually practicing engineering as compared to today's software development practices. Also software development as a field is far less mature than traditional engineering disciplines. That's not meant to be pejorative - it's just where humanity is.
This to me is one of the worst things about the Tech industry. You could be a senior engineer at Google and to move to a new company you will have to do the same algorithm interviews that New Grads are doing. It's a bad look for the industry. Like oh I see on your resume that you have worked for Facebook, Google and Amazon, well we better put you though an algorithm interview because you could have just been a false positive at all 3 companies. It massively devalues SWEs in an industry that we already know colludes to depress wages.
>This probably has more to do with Aeronautics actually practicing engineering as compared to today's software development practices. Also software development as a field is far less mature than traditional engineering disciplines. That's not meant to be pejorative - it's just where humanity is.
Not disagreeing but my point was that they have a far better performance record despite not doing the bizarre interviews that the Tech industry favors.
EEs definitely go through basic, advanced, and domain-specific circuit design questions in their interviews. And compared to LEETCODE they are much more difficult to prepare for too.
There is a reason why SWE is a fallback option for many other engineering disciplines -- resources like leetcode and low barrier of entry to learning make it much more accessible.
You're right that there are plenty of developers that have no experience with GitHub, but I'm sure there are far more who would struggle with simple theoretical questions and programming quizzes during the interview, because it's difficult to think clearly when you're feeling a lot of pressure.
I do like coding a lot, but these home tests can never excite me. I have a family. I can't do it after office. In office I want to work honestly on my job. So frustrating!!
It takes much longer than people assume to get a dysfunctional team member off the payroll. In most shops, new hires spend months in a (reasonable!) reduced-expectation state of on-boarding and ramping up, during which their performance can't readily be measured. It then takes several engineering cycles to figure out that they're not working out on a particular project, after which the normal response is often to try moving them to another project. You can be more than 6 months in with an employee before there's a confident assessment in engineering management that someone isn't working out. In many shops, at that point the next step is to begin the process of "managing them out" (often with multiple stages of PIPs), which includes a fig-leaf premise that performance can be fixed and the team member retained.
And that's if things are carefully managed. Lots of teams, even good teams, aren't carefully managed, and it can take more than a year to detect and propagate the signal that someone isn't working out.
The net result is that people can accumulate impressive-looking resumes without ever effectively delivering for any team, and because resumes get more impressive over time, the roles people get offered improve as well, compounding the effect.
I'm an evangelist for evidence-based hiring techniques and a fierce critic of standard interviewing practices, which I agree are dehumanizing and unreliable. But we started our hiring practice[1] not because we wanted to include more people, though that was a happy result of it, but rather because we'd been burned repeatedly by good resumes and strong interviewing skills.
At this point in my career, for a technical/IC role (really, any role), I'd be alarmed if I wasn't given practical tests to verify I can actually back up my talk. Their absence would make me suspicious (though perhaps not dispositively so) about the technical ability of the team I was considering joining.
[1]: https://sockpuppet.org/blog/2015/03/06/the-hiring-post/
Agreed, and let me offer an anecdotal, real-world example. Before I was in this racket, I used to be a land surveyor. Because it's both technical and licensed (and btw, I'm not going to make an argument for licensure, just mentioning it as a qualifier), you had to take a written test and pass a rigorous interview before you were accepted as an apprentice land surveyor. The test involved trigonometry, plane geometry, algebra, property law, and basic statistics. The sit down involved attempts to surface red flags all the way down to testing if the candidate was afraid of dogs.
And after passing that, then you were hired on as an apprentice which meant that all your work had to be reviewed and signed off on by a licensed PLS while you learned on the job and could eventually take the PLS exam yourself.
Too often in software engineering, a candidate can jump through the hoops and get hired but even if their role is junior, they're essentially on their own except for code reviews. There's very little mentoring, because senior people are neck deep in work and nobody's got time for that. The company has to be committed to mentoring and carve out space and money for it in order for it to happen.
We would be much better off if the craft of programming was treated like other trades, requiring training and formal apprenticeships, with the necessary mentoring and signoff.
Programming is to CS as plumbing is to fluid dynamics. Knowing how to write a bubble sort is an irrelevant skill.
Knowing how to use the standard library in the environment is much more relevant.
Older developers come in two forms. Those who aggressive learn, have matured beyond “just developers” and those who are set in their ways. I’m 45.
I'm still buying a copy of cracking the coding interview and doing leetcode in the weeks leading up to applying for precisely this reason. It's dumb but I'm not going to give up an otherwise promising opportunity in protest, although in the interview I might politely ask (after solving the problem) what relevance it has to the job, just to make the point. ("I thought I was a applying to do X, and this doesn't seem to fit the job description. Does the job involve a lot of work like this?")
I needed to represent the shortest path between nodes on a map, so I needed the internal data from the algorithm, thus the need to whip out my own version.
But just as higher level Software Engineers rarely have a need to implement recursion/graph traversal/etc, in embedded its even rarer due to the relative expense of such operations. I'd say knowledge of signal processing and how to power/performance/memory-optimize code would in general be much more valuable.
1. The size of the data is often much smaller - the savings from using an nlogn time algorithm vs a n^2 algorithm are much smaller when n=10 versus n=10000. There's a good chance that the constant time setup will dominate for those smaller datasets.
2. To a certain extent, embedded systems don't need to be as fast as possible - they just need to be "good enough" to meet their deadlines so that the rest of the system functions as expected. It'd be nice to be able to compute 1000 CRCs per millisecond instead of only 10, but if it already takes 1 millisecond to transmit the CRC over the serial bus, it's not going to make the system substantially better at its job.
So yes, there is a lot of handcrafted code, but quite often it's just implementing your standard naive brute-force algorithm with minimal code size, minimal memory overhead (preferably without dynamic allocation), and well-understood worst-case latency.
Programmers will endlessly prematurely optimize. I'd much prefer to see simple code that can be understood and optimized where needed, rather than squirrely code that runs fast but is unmaintainable.
It is common for embedded software engineers to come from electrical engineering, and they are maybe less likely to ask CS-type questions.
Best to be prepared.
Out of nowhere, after I thought the interview was over, one guy kept drilling me on that type of low level stuff that had nothing to do with the job just to prove how smart he was. I remembered all of it thanks to a decade of dealing with it and too much time on comp.lang.c but it was still off putting.
I’ve since cut all of that stuff off my resume.
If I'm already 100% sure I'll recommend a hire, it's not rare for me to go off-script and dig into random stuff in the candidate's experience, first of all because I find it interesting or fun to talk about, and second because having deep knowledge about anything is positive signal, regardless of whether it's useful for the job.
I have definitely been in this scenario and asked people who had relevant background to work out low-level performance details of their code. I hope they didn't think I was "drilling" them.
I think he asked me something about the difference between undefined and implementation defined behavior and the an example of each. This was in 2015. I had the same question when it was relevant for an interview in 1999.
I’ve geeked out with another hiring manager at an interview about some old 8 bit coding I did but that felt different.
The last thing I would want to see is lines of code on a whiteboard. I'd want to see how they'd structure their code, not whether or not they remember C/C++/Rust/language-de-jour syntax.
All of the proposals for questions that I've seen in this thread are for basic programming skills. Not whether or not the person knows the environment we are hiring for.
I highly advise you don't do this, because I don't consider it polite. Everyone tries to be objective in final review, so don't jab them with something snarky they'll remember about you while discussing your performance.
It's an exercise, it's artificial. Can you convert an abstract idea into code. Yes there are bad questions and bad interviewers, that doesn't mean wanting to see you write code is bad. You don't want Microsoft's new "real world, from the job" style questions [1] because I've had a few of those and they're misguided. Way too much internal domain knowledge is missing and I spend too long trying to get it out of them.
That said, DP in a phone screen for devops is a little tough.
[1] https://www.businessinsider.com/microsoft-new-developer-inte...
Some managers embrace the notion that all software engineers are basically the same and you can shuffle them around outside their area of expertise and then complain when efficiency goes down. Now granted there are probably more strategic ways to ask about that then the direct method I originally posted, but if I'm interviewing with one of such managers, I want to know.
I just want you to have the best shot at this job. Be as critical about the process as you want after you get it.
And you're right. 9/10 times it'll just be a fizzbuzz proxy and not worth the risk of a direct question.
Also, it reeks of "justice driven opinion having" which is scary beyond reason for an employer, e.g. "They didn't ask for this feature the right way so I'm not going to deliver it/deliver it in a way that makes them unhappy."
Interviews go both ways. If you have a problem simply saying something like "nah, these are just screening problems, it's hard to come up with good ones" then it sounds like there's deeper problems here. Like, the company having issues recognizing even obvious flaws in its own processes.
Someone who gets defensive about an honest, neutral-tone question about their interview process is likely going to be defensive about more important decisions, which means they take feedback poorly and are likely a poor leader. This is assuming I'm talking to a hiring manager and not an HR rep.
I don't apply to missions I'm not willing to work overtime for. If I'm going to make sacrifices for whatever my employment does, I require that my leadership shares my commitment and is fundamentally competent such that they don't needlessly waste my sacrifices. In any interview I'll be probing for that as respectfully as I can, just as they'll be probing me for technical aptitude and whatever character profile they're looking for.
The mere fact that you think you can neutrally ask a question like that is a red flag.
As for "role" and "expert", you seem to think that in an interview my job is to be subordinate. Quite the contrary, at present I'm not unemployed and my bills are paid. I'm looking for an upgrade, not a necessity. So in an interview I'm still deciding if the company is worthy of my subordination/my making them more money. I may not be leadership, but I can pick which leader to support. Leaders who are oversensitive to basic, polite questioning during a job interview clearly don't want/need help that badly.
Now once a subordinate, obviously there are times/places for questions and times/places when they are not helpful. But if you consider questioning decisions in general "poison" for your team, and you fear your team turning into a "snake pit" if someone starts asking questions, then that points to a severe lack of trust and/or poor communication between you and your subordinates. Either of which would turn me away even if you offered to double my salary.
Asking questions doesn't create a snake pit, not being straightforward with your opinions does. Asking leading questions that force people to not trust your motivations will certainly create a toxic work environment.
Related, I don't think you actually believe I was genuinely putting forward the idea that you shouldn't ask questions in your interview. I have some not-great theories about why you chose to do that, but this probably isn't worth the time it'd take to figure it out with you...
As for "knowing better", in the past I occasionally have known better than some of my bosses, as born out in money lost (once to the tune of millions of dollars) doing it their way until they (miraculously and of their own accord, of course) came around to doing it the way some of my co-workers and I had been promoting. Was a minority of cases, but often enough that I don't assume competence or honesty from strange managers without at least some indirect verification.
As for not asking any questions in interviews, I didn't mean to insinuate you meant that exactly. But from your statements it appears you would have issues with non-technical non-job-specific questions, because I'm an engineer and such questions would be "taking a moral position beyond your role on a topic you're not an expert in.". Which to me sounds like "Engineers shouldn't judge businesses beyond their immediate role because they don't have business degrees". To which I'd say I don't need to be an "expert" to judge interviewing techniques any more than I need to be a licensed auto-mechanic to change my oil and brake pads. Likewise I've learned the hard way to examine a business's financial incentives/disincentives, business model and direct competition more closely than the job descriptions, because those factors will determine the nature of the job I'm applying for a lot more than any given mission statement or brief description in an interview. So I may ask questions about those topics as well, despite being "beyond my role".
Since you value direct honesty about intentions: this conversation is beneficial to me in forcing me to explain/defend these perspectives in greater detail than I usually have the opportunity/need to. So for whatever it's worth I thank you for that. It's clear we both come from very different experiences in approaching this scenario, and I'm sure your experience bears out your arguments just as mine does my arguments. I doubt we'll come to an agreement here, but part of me wonders how I'd do in an interview with you if I wasn't aware it was you and you weren't aware it was me, from this thread. Probably never going to find out, but would be interesting.
What you end up with is someone who is either a mid-level engineer and almost totally green to recruiting or an HR drone with zero technical skills who is liable to believe a bunch of bullshit. Both tend to ask stupid interview questions they learned from a book because they’re out of their depth in some way.
This is not a situation unique to software either; it affects nearly every professional specialization (especially ones where the pay gap between an HR role and a functional role is as big as it is in tech). You’re right, it makes zero sense to put your interviewer on the defensive — being likable is almost always more important than being good.
In a way, it's nothing more than studying for some goal you think it will improve your current situation, and the good news that it is pretty much a solved problem, with plenty of information online that can be used to practice.
Just take into account that for an embedded position you might actually be asked more domain-specific questions.
I wish you the best in your seearch!
Don't bother with books about how to get a job. You're wasting your time.
Instead, spend more time with books about hiring. How to run an interview, what to look for, ect.
The reality is that companies mostly want to hire someone who can do the job and behave in a team. (There are some exceptions, but you probably don't want to work there.)
At the end of the day, doing lots of leetcode exercises doesn't prepare you on how to convince someone in an interview that you can do the job. A few leetcode questions might help build your confidence, but that's about it. Really, all you need to do is play along and evaluate the people who are interviewing you.
BTW, what I usually tell candidates goes something like this: "Look, we only have an hour, so these questions are contrived. We don't write code like this on the job. The best way to succeed is to play along and answer the question." And then, as I get into the questions, I sometimes add things like, "This isn't a JSON versus XML question" and "If you can use async-await, we wouldn't have a question. I'm trying to evaluate..."
I don't ask technically challenging questions. I'm more focused on comprehension in a technical discussion, and a basic comprehension of concepts that are critical for the product that I work on.
Also, yeah probably no dev-ops position really requires knowledge of DP. But they generally do require programming knowledge and probably an algorithm here and there so it's not entirely inappropriate
LeetCode questions test for other things than "basics of coding".
https://www.glassdoor.com/Interview/Palantir-Technologies-De...
Yes
> probably an algorithm here and there
No
Even Amazon doesn't ask about algorithms for DevOps. When I interviewed with them for a DevOps position, they asked me to write a Python script to parse logs files. They also asked about various linux commands that you would use to work with those same logs. Pretty relevant. But the interviewer never once asked about algorithms or anything like that.
My current company gave me a take-home test to build out a solution that deploys a code package. Best interview question/task I've ever had. It was hard, but I got to show off my real skills.
Companies that ask leet coding questions for DevOps probably have no idea what they're looking for and probably don't know what they're doing.
I think it's a more a case of cargo culting rather than not necessarily knowing what they are doing, although that might also be the case in some instances.
My last in-depth interview was a take-home/collaborative project (paid) for a smaller company here in Canada. It was a great experience. So good on you.
Someone I know did an AWS interview round during business school, and they do this thing called "bar raising" (IIRC) that involves one of the interviewers literally doing something unpleasant to see how you react (e.g. picking their teeth in the camera, cutting you off when you speak). Very odd.
I'm not saying your friend didn't get a bad interviewer. It's just an interesting demonstration of how risky it is to generalize from anecdotes.
I'm guessing the way Amazon interview for business roles is somehow different than how they interview for specific team positions. As I understand it, MBA hires aren't filling specific gaps on teams, since the hiring process is so long and involves an internship. There wouldn't be a "team" to lower the bar for, in other words.
I also just spoke with my wife and she re-confirmed it. So I guess I'm wondering what these people who I respect are experiencing...
Of course I hate them, but what other choice do I have?
The biggest problem is that the questions are so random and span almost anything.
Will I get asked:
a pthread question?
implement mergesort or quicksort?
implement a Read/Write lock?
implement a smart pointer?
a bit manipulation question?
topological search?
dynamic programming?
graph question?
backtracking question?
What the default timeout for TCP/IP is?
a Java internals question?
best practices about Cassandra?
N-ary tree search?
linked list?
b-tree?
Paxos vs Raft?
What is the access time for an SSD vs spinning disk?
How does Hadoop work?
Design Netflix
various random programming questions, as evidenced by Leetcode's 1000+ questions
It's impossible to know everything and yet this what people expect and ask. I've been asked those questions above over the last 6 years. It's utterly insane the expectations that interviewers have.
I personally also interview candidates, I've done hundreds of interviews throughout my career. These days, I do about one interview a week, and often 2.
My coding question is a simple recursion question, with a for loop and some basic knowledge of data structures. It's not hard, but I look for perfect code. If you can't code a bug-free 15 line function in 45 mins, that's my red flag. I don't think that's unreasonable. As long as the person asks good questions, we work together well, and they can code in a sane way, to me that's a pass. Interestingly, 70% of the people fail the interview because they can't code properly, even after the phone screen. I think because people are expecting a hard leetcode algorithm question, they can't think properly when someone asks them a simple question.
Exactly. I have no problem coding up a solution if I have an hour to do some research. Either give the topics ahead of time or ask it as a homework assignment.
If the job requires hurry-up coding solutions to whatever crap falls on your desk without any design, research or discussion of alternative, the quality of their product and codebase probably reflect this.
Not really though. They always manage to fill the position. Interview’s see the system as unfair because they think their interview happens in a vacuum. You get x questions answer all correctly and your awarded with a job. So naturally you complain when you feel half the questions are too advanced or that more brain understanding is enough. In reality the interviews are just a sorting, and when everyone can answer the relevant questions you start asking harder ones because “we picked a candidate at random from those who got the questions right” is just not a satisfying endstate.
That's just silly. If they can answer the relevant questions, they're qualified period.
Imagine a single resturant forced to hire 200 chefs because each of them managed to live up to the minimum qualifying criteria for the position...
What does it mean for a software developer? Spend a few hours a week doing leetcode questions. They're really not that hard.
Suggesting that they do would not address a glitch in the interviewing process but rather try to accommodate it at the cost of the interviewees’ time.
If you don't put in the work, someone else will.
These are high paying positions. You can't just expect to lazily stroll in.
My concern is about the success of the company I work at. My coworkers tend to be excellent when the bar is a tough algorithms interview, but they can't hire enough people. I'm a particularly demanding interviewer. I've rejected candidates who failed miserably, then later learned from my excellent coworkers that the candidate I rejected was excellent at their job.
I've also referred an excellent coworker from a previous job, only to have them fail the interview process. My current company would have benefited greatly from that hire, but instead rejected a talented person for a role they were having trouble filling.
If a company is having trouble hiring, it needs to at least get these excellent people that are being filtered out. Lowering the bar to let these excellent people in also lets in much less qualified people. Instead, the filter needs to be changed to let in people based on how qualified they are rather than how good they are at algorithm puzzles.
Successful companies need more than just people who are good at algorithms like me and studious people like you.
But if in your own words “you studied for an interview”, it’s not exactly showing that you have a special skillset if all someone has to do is study for a month.
If you’re an experienced software engineer and companies aren’t beating down your door and you’re still randomly applying for jobs and jumping through the same hoops as someone just graduating from college - you’re doing it wrong.
We're irritated with the current interview practices, because the gauntlet that many of us have to run is not only wasteful, but it's also harming the profession. If one has to jump through the same old hoops their entire professional life only to be thrown out at 40, what's the point of building a career in software engineering?
This is accepted blindly as dogma in our industry but in my experience it's only the case if the company is dysfunctional and management is too weak to let poor performers go quickly. Being an effective leader means being able to hire and fire quickly. If you're open and honest about it, you don't have to do as much as much intense vetting up front which costs your team's time and filters out potentially great hires.
The net effect it has is that firings are uncommon, and people hang around until the next round of layoffs, when a reason will be found to lay them off.
Citation needed. I have many years of experiential evidence that suggests otherwise.
I've interviewed dozens of people, and you can easily root out false positives by asking about their prior experience. Simply probing about things like design patterns to challenges with Javacript front-end coding is enough. If the candidate can go into explicit detail about past projects including bugs they fixed and limitations they came across in a Angular 6 project (as an example), then that's usually a good sign they know what they're talking about.
I already had an offer by then but I did the interview for grins and giggles and didn’t try to remember what a merge sort was. I actually wanted to test the culture of the company. If they ask me to write a merge sort and I don’t know the algorithm, will they describe it on a high level or will they look at me derisively? That would tell me a lot about the company.
Well anyway, the guy was nice enough, I did the mergesort on the board, got an offer but decided if their interview for “senior engineers” consisted of the ability to do a merge sort instead of deeper architectural questions, that wasn’t the company I wanted to work for.
When I conduct interviews, I have a simplified non working (but compiling) class that is a sample of the types of problems we solve everyday with a set of failing unit tests. We pair program and they have to see if the tests pass.
There is no persistent storage in humans - people forget things, and it is quite annoying to do this again and again for every new job.
People say this, and then they whine about how they can't find good candidates, can't find diverse candidates, etc.
They could, of course, study interviewing, treat it with the same level of seriousness you might apply to other skills, and actually learn how to evaluate people with a pretty good level of accuracy so you can avoid both problems.
DevOps are teams, not people.
"We'll use python on GCP with PostgreSQL as a backing database"
Well, then you need an expert in GCP, Python and postgresql. This can be as many people as it takes to have those skills covered.
As for responsibility, well, that goes for where your skill is. If you're the resident postgresql expert then you're expected to be responsible for supporting people with it.
It's likely you'll need to worry about patching the OS and stuff like that, but I don't think you need a dedicated person to be responsible. The point of DevOps is mostly to bring down the silo'ing, not to eliminate a particular role.
But it certainly appears employers are salivating over the idea of hiring less specialists, or simply rebranding sysadmin to "devops".
I'd rather say the point is the opposite; devops teams are (even moreso than agile dev teams) heterogeneous (or “cross-functional” or “multi-disciplinary”).
There's basically a continuum where assembly line analysis / dev / test in separate teams (and security, operations are separate higher level organizations with the technical area, and business is separate from technology) is one extreme, then agile dev teams unifying the analysis/dev/test functions, then devops, then devsecops, then maybe the other extreme is some thing that goes beyond devsecops where also domain expertise (and not just the system analysis skills that interface with domain experts to elicit requirements) are an organic part of the team.
But each step along the line does increase the expected range of skills of an individual team member, because you need also to avoid any team member being a single point of failure and you have to keep teams in manageable sizes (e.g., 5-9).
Whups, I used a very wrong word here, I meant to use the word "heterogeneous". My apologies :S
Hire the software engineers with an appropriate interview process covering development practices and coding.
Hire the system administrators with similarly appropriate interviews.
Hire the database administrator with their own appropriate interview.
Is this that hard?
Not every person on the devops team needs to be a perfect 50/50 split of dev and ops.
Good DevOps engineers are above everything else good developers. Those kind of developers like to control every aspect of the final product so then they enter system administration, networking, deployments, metrics etc.
On the other hand there is DevOps culture and teams and that is something else and can be reasoned about separately.
There are software engineers, there are deployment automation engineers and there are people that do both to a greater or lesser degree.
From a recruitment point of view, it seems devops engineer = deployment automation engineers
Thus the expectation from a coding POV for a devops engineer is low, i.e. only scripts.
As ever, there are nuances here that can't really be captured in a post but you get the idea
(BTW I have the exact opposite opinion, I think devs should stick to code and not try to setup infrastructure for themselves)
You don't need a cloud to implement DevOps culture to development process, yet you need people who know their shit when they do Ops work (i.e. Poor Yorick incident in Gitlab).
It's important to have someone who is able to stop and ask for help instead of going further down a blind alley. I've found it filters out a lot of the know it all types who never mange to get anything done.
I doubt that was the intent of the OP's interviewer, but you never know.
> Since I've started using it (admittedly my sample size is small) I've found some very effective programmers,
This is not the only way to find very effective programmers. You should level up your own ability to discovery this in someone in a non antagonistic and more straightforward manner.
If candidates come out and say such when I ask them questions about it, I'll just move on. I won't hold it against them unless it happens excessively.
I don't really agree with him and just kept doing it though and things seems to be fine.. If anything I would say that interview was successful for both parties because we probably wouldn't get along and that is totally fine.
He said "Don't just tell me what you don't know, tell me what you do know" or in other words, don't just say "I don't know" but say something like "I don't know, but based on what I do know this is what I think the answer would be and this is how I would figure it out".
It really helps that everyone forgets what people say before the "but" and you can help get across the idea that while you may not know everything you know how to figure things out.
It's a trick question because you want the interviewee to answer something different (I don't know) than what you're asking them to answer. Confident applicants might feel up to saying they don't know, but nervous people who are very self conscious in an interview setting may respond completely differently than they would if they were asked the same question on the job by the coffee machine. Their mind would be racing "oh no, I'll look so bad" and they'd go into full panic mode.
Don't ask trick questions. It's smug and useless. Plus you'll be missing out on great smart colleagues who happen to stress out about job interviews.
The ability to ask meaningful questions is a big part of any programming job. And (in my opinion) harder to learn than any particular technology.
If you want to see them ask questions, give a half explanation of the problem at hand instead and guide them to ask more detailed technical questions.
Be an ally, not an adversary, and you will find talent better than if you push.
To be specific, I do a lot of things up front in an interview to help the candidate. The most important is informing them that they should treat technical exercises as a team exercise where we are teammates working together. Furthermore, I declare that there is no penalty for my assistance during the interview. The idea is for us to work together to solve a problem, my interest is in how they think and work through the problem. Furthermore, I explicitly tell them for discussion on prior experience and technical facets of interview that they are welcome to indicate they do not have an answer to my question or are unsure what's next.
All of this combined creates a very pleasant experience which is an important part of my goal as an interviewer. I'm not trying to embarrass people, though I am trying to make a informed hiring decision which is a very expensive endeavor for my company. Therefore, beyond letting candidate enjoy themselves, my main focus is to ensure that I have sufficient confidence to hire or be willing to walk away due to lack of conclusive data.
As Jeff Bezos says, the older he gets, the more he realizes he doesn't know.
Most people have no issue with saying "I don't know." I've worked with people that can't say "I don't know" - they are incredibly toxic.
I don't think this is true only in interviews though. I have worked with people who can't seem to admit they don't know something and they're difficult to work with.
It's completely fine if you can't do something. What's not fine is that you fail to communicate it to those around you and create false expectations.
That being the case, I think it's fair to score a candidate negatively if they aren't able to say they don't know something.
with hacker news so utterly convinced that employers have literal powers of life and death over their employees, when do you expect the potential employee to regain the ability to tell their employer that they don't know? after they've quit their previous job and moved across the country? a week after that? six months? maybe after they've got some kids?
In a way forcing the other party to give up with an "I don't know" is humiliating, and graciously allowing this answer doesn't make things better. You'd put yourself above the other person.
Imagine you'd interview someone you truly respect, a luminary of Open Source walks in. How would you go about the interview then?
I think a great interviewer can make the interview an enlightening and wholesome experience for both parties - while easily identifying the exact qualifications and the fit of the candidate.
As for interviewing a luminary of open source what would be the difference? I imagine they (much like everyone else) don't know lots of things and are probably comfortable saying so. Their answers to how they would go about finding an answer would probably be wonderful.
Of course it depends a lot on the overall tone of the interview, and as I read more I'm sure you're doing it very well.
If I were in an interview that was presented as a zero sum game I would walk out too.
I mean, like, yeah they did. They either got the job or they did not.
[0] which is not really super relevant to this discussion, I know.
A big part of interviewing candidates well is breaking down that dynamic. There are many ways of doing it - I get up on the whiteboard first and write out my problem, and stay there until they’ve explained themselves and I need to get back to my notes so I don’t forget stuff. I phrase questions like “I need help implementing this function...” or “how are we going to solve this edge case”.
I don’t think either side of a tech interview is easy. The problem is only one side is evaluated.
Not yet, but the purpose of the interview is to figure out whether we should be. (And I disagree that it is one sided; this whole topic is about an interviewee's evaluation of the process.)
I'm not sure I agree here. Granted I work for quite a small company, so it's pretty different than working at a company with a significant reputation to help candidates make decisions. But when I interview candidates I'm very aware that they are evaluating me whilst I'm evaluating them.
I actually think this is a good thing. It incentivizes me to make the interview a more pleasant experience, and perform it in such a way that it feels collaborative, like a real work situation, and not as standoffish as some interviews can be. That said, we still do ask hard technical questions solved on whiteboards or in code, and setting them up in such a way that the candidates feel comfortable asking questions on things they don't know / are unclear about is definitely challenging.
To be honest, something I think we (as in the software community at large) lose sight of some times is how unique and challenging performing interviews is. We see stories like this pretty frequently, and we're used to the idea that being interviewed is a fairly distinct skill, orthogonal to what we usually perform in our day to day work life. Something that's talked about a lot less frequently (understandably, more of us are getting interviewed than performing them) is that interviewing others is also a separate skill, and one that really isn't practiced or honed enough in my opinion.
I tell all of my friends that it's your job to interview the interviewer as well. The company is about to get 8 hours of your day for the next few years. figure out how you can figure out if it's the right place for you.
Obviously, if you are in dire straights and you need to take what you can get, sure, but the questions are still relevant and will prepare you for whatever poo you might find yourself stepping into.
I don’t want to hire someone who isn’t going to admit the limits of their abilities, for whatever reason. Could be smugness or lack of confidence, either way it’s a bad signal.
its not a bad signal. to be honest its the smartest thing to do (like guessing on the GRE), especially since most people ask questions they expect the right answer too.
the only bad signal I see is in you. I would wager to say, in most situations in life, don't play tricks, just ask the question you really want answered.
This part is crucial. If an interviewee is pushed to the brink but doesn't say, "I don't know," that does not imply they will not say, "I don't know," once they've secured a position.
You're only going to be selecting candidates who can handle pressure -- not skill or ability with this.
How is being asked something you don't know the answer to being pushed to the breaking point? Do we expect to learn nothing on our jobs?
Back in the day I had an interview for a business position where the interviewer's personal printer mysteriously started printing documents part way through. I'd just read "What Does Somebody Have To Do To Get A Job Around Here?" which listed that among other tactics a company might use to see how you handled distractions. I've suspected it was on purpose ever since. Even were it intentional though, it would be more of a gimmicky ploy than an adversarial thing to do.
An interviewer should be able to modulate the difficulty of a question without making you feel uncomfortable.
Also the job isn't done in a vacuum - being able to remain calm when you don't know the answer is an ability all its own. Interviewers shouldn't be creating artificially stressful situations but you've also got to be able to manage your own stress response!
Truly, the best interviews I've had have also been the hardest. Different strokes for different folks.
"just because I can". To me this technique comes off as condescending and manipulative, and smells like a power play basically. Not to mention - the better candidates will pick up on it, and your company will eventually get a reputation it doesn't want to have.
And really there's no need for it. Time and time again, I find that just by having a normal technical conversation with someone, on virtually any topic that they claim to have expertise in -- they will show their strengths and weakness (and propensity for BS, if any) very, very quickly.
And it would be manipulative if there were some answer I was going for that wasn't true - I'm trying to elicit an honest response.
When I'm interviewing someone I will often preface groups of questions with "have you worked with {x}?" or "are you comfortable answering questions about {x}?" before asking the actual questions covering that area. A simple intro question like that goes a long way towards keeping an interview friendly. The interviewee can say "no" to a small-sounding question without feeling like they're embarrassing themselves.
For example, they may be comfortable with Python but not super-comfortable with the GIL/multiprocessing/multithreading. If I ask "have you worked with Python's multiprocessing or multithreading before?" that comes across as much less adversarial than asking them to describe Python's GIL and forcing them to say "I don't know."
Doing an intro question also allows them to give a partway answer like "I've touched it before, but I wouldn't call myself an expert". If they say something like that I can re-focus my questions to another area, or just ask higher-level questions, rather than hitting them over the head with a deep question about something they've just told me they're not an expert in.
You do want people to be honest about their own limitations, but you don't want your interviewee to think that your organization is driven by macho posturing about being King Nerd.
The answer I’m looking for is “I ask for help.” Or “I talk to my manager.” They don’t fail if they don’t answer it that way, but it’s a good starting point for a conversation about why they didn’t say that.
For me, interviewing is more about getting to know someone and how they think than checking off various skill boxes.
“DevOps is a set of software development practices that combines software development and information technology operations to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives.”
I don't think it would be appropriate to call someone copy-pasting templates, manually adding VMs, etc. "devops"
In my experience a “good” SRE, devops engineer, or software engineer is difficult to distinguish in skills. It’s more about responsibilities. This is especially true of the folks I’ve met from FANG.
https://cloud.google.com/blog/products/gcp/sre-vs-devops-com...
The consequence of 'get [anybody with a pulse] into coding' is to help increase the supply of people with some coding ability. Out of curiosity I just searched for 'QA job listing' and found stuff like this [2]. A 'senior QA engineer' requires, among other things, a masters in computer science alongside a willingness for unanticipated travel/relocation for "long and short term assignments at client sites." Starting salary? $73k. That's probably an H-1B angle-shoot (H-1B employment requires not displacing American workers, and if nobody applies you're obviously not displacing anybody!) but even then they still have to be able to argue that their request was not entirely unreasonable.
[1] - https://www.inc.com/suzanne-lucas/why-that-barista-has-a-col...
[2] - https://www.indeed.com/rc/clk?jk=5d42d374514328da&fccid=d045...
If you're going to need to dig into the source code of infrastructure pieces to find/fix bugs or write your own infrastructure pieces then a SE interview makes sense. If you're doing things at scale or using newer technologies then these come up often enough and resolving them can have significant business consequences (not getting into if choosing that path was a good business decision or not, separate topic).
There are also companies which want DevOps to know SE simply because they believe that such people create better infrastructure. Better in this case means more automation, code versus todo lists, understanding the SE users, etc.
It's pathetic employers keep demanding more skill and free work for the same pay.
In my experience DevOps pays better than SE (and much better than sys admin) because it requires a diverse set of skills. DevOps in that context is not just a new term for sys admin.
DevOps is sysadmin with more scripting, more infrastructure as code, and more expectation that you're an underpaid SRE. It also seems like DevOps role demand more unpaid on call responsibilities, where this is usually not the case with traditional sysadmin roles (or it's spread across a much larger team, negating the pain). YMMV.
Disclaimer: 20+ years in tech, the majority in ops roles.
Another benefit is, its streamline job searching. You practice once and you can use the same skill to interview for many companies.
1. being able to solve LeetCode tasks doesn't mean you're a good developer
2. not being able to solve these tasks doesn't mean you're a bad developer. So we have a tool that has a certain (IMHO high) rate of false positives and false negatives.
3. people start realising that LeetCode is becoming a go-to tool for recruiters, so instead of studying to become better developers, they study to become better at solving LC exercises, which increases the rate of false positives
4. with developers becoming better at LC, the bar is set higher and higher (because people spend time learning LC), so people who do not study LC exercises become far behind those who do, which also increases rate of false negatives
5. repeat points 3. and 4. until mastering LC exercises becomes more important than being a good engineer
This is a ridiculous trend which in my opinion only emphasises how screwed up is hiring in our industry.
If the job requires algorithm wizardry, I see the benefits of LeetCode-style interviews but many jobs are not like that.
You do it in addition of learning skills you use on your job.
For the me interviewee, it also make the interview process more streamlined, scalable and predictable. I practice once and I can use it for many other company, thus save my time.
At the margin, companies will lose out on qualified candidates that have better things to do with their time than practice algorithms and hire unqualified candidates that got lucky with their Leetcode practice.
Most companies have cargo-culted the programming interview, even when they don't have the same amount of demand from candidates.
Should movements such as 'get [anybody with a pulse] into coding' succeed, you can expect to see an ever increasing absurdity in requirements. The reason is not to try to be elite or whatever, but because the reality is you have to get rid of of > 90% of applications, for larger companies it may be > 99%. And so as the number of applications increases, you get an ever larger quantity of people with at least the minimum requirements and so you are forced to be increasingly selective.
Does passing some leetcode task mean you're a good developer? No, of course not. But if you take 100 passable developers and you're looking for the best, are they more or less likely than average to be able to pass such a test? It's the same reason that even though a college degree has absolutely 0 relevance to serving coffee, you're increasingly seeing companies "preferring" college degrees for their barely above minimum wage baristas. [1] If you take 100 potential baristas is that best of the best of those 100 more or less likely to have a college degree? Even though it has nothing to do with the actual job, it's just a decent way to filter 100 down to 1.
[1] - https://www.inc.com/suzanne-lucas/why-that-barista-has-a-col...
Is the process great? Absolutely not, and I'm sure that these places miss out on qualified candidates because of the process and hire some people who gamed the process. Things like degrees and LeetCode tests and other screening methods are just ways to reduce cost and risk to get that "good enough" threshold.
Understaffed projects might lead to failures, people quitting, expensive consultant aid etc.
Hireing a bad programmer and two good and ending up giving the poor one easy tasks or letting him go might be cheaper than hireing two good more slowly.
In general I feel the recruitement process is so random anyway that employers should just get it done and pick someone after throwing darts at a billboard with resumés for picking some to interview.
In my day to day I am working on mission critical applications, creating web applications that rely on legacy databases, writing SQL SPs. I'm decent considering my short career, i try to improve on every previous project. However, when I go home I'm not wasting my time grinding leetcode, even though I was top of every class in college, algorithms, data structures, theory of computation and so on. I'm competent and pick up new stuff very easily. I tell recruiters I will pass on an interview if they are doing leetcode/hackerrank. I'm okay to solve some coding problems that they have day to day, talk about architecture/design, talk about what I'm good at and where I want to improve. I also wish other people would stop grinding leetcode and that everyone was realistic about the job at hand. Most developers won't be writing the most efficient code, but start easy and refactor if certain part of the application becomes a bottleneck.
Half of those questions are based on sophomore-level data structures and algorithms classes, the same concepts that operating systems, compilers, and most libraries/frameworks are based on. I would certainly hope that an experienced developer could effortlessly relearn basic concepts taught to college sophomores long enough to pass an interview.
Don't assume something is easy or straightforward for everyone because it was for you.
I switched tracks to the normal software engineer with only-leetcode interview questions. Spent 3 weeks studying and last week I went in and had an easy time because it was exactly what I studied for. Also applying to a few other dev shops was effortless because they asked the same stuff.
Doesn't that support the argument a bit? Its "not that hard" -- so even someone who _doesn't_ have the requisite experience could learn it? So its just another hoop to jump through, _before_ they validate whether you are a good fit for the job in the first place. That's what people are complaining about.
>so even someone who _doesn't_ have the requisite experience could learn it?
I consider it a plus. Every company use different technology and different way to do things. Not to mention it keep changing all the time. With a standardize test, even though I don't have the experience with the technology they use, i can still have have a shot.
> Every company use different technology and different way to do things
To a small extent, but mostly untrue. If you are an expert UI programmer, you should be able to move between disparate UI technologies relatively easily.
> With a standardize test
People aren't upset with the standardized test part. They are upset it (frequently) tests things that are not relevant. Hard things at that. There are an extensive number of practical, language / framework agnostic questions you can ask about UI or back-end programming that will easily tease out quality candidates. Instead they ask you merge sort. Hence people complain.
To some extent but the topic is still relevant, leetcode topics are algorithm, data structure, system design, etc.
Again whatever interview scheme you devise nothing is perfect. To me at least the pros outweigh the cons.
If their onsite has more leetcode coding questions, then the interview served its purpose.
What does concern about some DevOps interviews is the sheer level of knowledge some companies expect up front. I brushed on everything from the Linux kernel, ops techniques, distributed systems and programming to prepare for an Ops interview.
We (my company) were interviewing a DBA. He listed C# on his resume. We develop C#, so I was brought in to evaluate his C#.
He flat out refused to answer my questions.
(In general, if you don't want to work in XYZ, or don't want to be asked XYZ in an interview, don't list it on your resume. That's why I don't list Installshield and Visual Source Safe, even though I've done them and can do them.)
I’m pretty happy with how my company does it now. A code sample on a fairly simple problem designed to determine how their pull requests would look as well as how much direction they need. On-site is a series of design discussions with algorithmic questions about data structures to use and the tools / services that make sense. Primarily meant to hire senior engineers rather than juniors, but that’s our objective now.
> So basically it was a variation of the exact change problem (I realized afterwards). You had to validate a string of size n. 1 chunk of 8 characters could have 1 format, 1 chunk of 16 characters another, and 1 chunk of 24 characters another. You essentially had to determine if the string was valid. So in other words, find a combination of these sizes in which the string could be validated. I had 30 minutes
> It can be a string of size n. It needs to form a valid combo of 8, 16, and 24 char strings - each of which have an expected format. Result is T or F
This sounds like a pretty basic parsing problem to me, not dynamic programming, and likely to be a regular language. There’s a standard tool for this sort of thing, a “regular expression”. Not recognizing this much is a huge red flag for any kind of systems/integration role, like DevOps.
The answer here is just matching against this regex, with “fmt1”, “fmt2”, and “fmt3” replaced by regular expressions for the different packet formats:
^(fmt1|fmt2|fmt3)*$But this is a dev perspective though. Am assuming a scripting answer might be good if the candidate knows about the actual regex behavior for devops.
I think it's great that the interview process is so learnable. Why wouldn't you be happy knowing pretty much what to expect going in when for most roles that isn't the case at all?
1) if the employer thinks DevOps is all about knowing how docker and k8s work, it's not a DevOps role (which doesn't actually exist by the way). a lot of employers think DevOps means "sysadmin in the cloud that can follow an sdlc", so you have to ask lots of questions to find out just wtf they're actually looking for.
2) if they ask stupid questions and generally don't seem invested in finding the right candidate, it was a crap job and you dodged a bullet.
E.g., I usually put a database question on interviews, for the purpose of seeing what and how much the candidate knows about databases.
I also like doing a design question regarding handling larger data and doing back of the envelope complexity analyses of the solutions they come up with. I.e., what I expect a colleague to be able to do when we're working at our desks.
I also expect them to work well on a team, especially in design reviews and code reviews.
The main difference between a software engineer and a devops engineer isn’t writing code, it’s the kind of code they write.
The other main difference is on your team. At my company we would expect that from a software engineer working on the platform, creating tools or shared libraries. A DevOps Engineer at my company would handle the orchastration of shared tools, (i.e. where is Jenkins hosted (VMs or Docker), how it scales, upgrades from Nexus to Artifactory, setting Dynatrace on prem with hooks into multiple cloud providers, the list goes on forever.)
Let's not assume all companies are the same, when probably half of DevOps roles are not about programming in an OO language and you can't easily or usefully unit test something that is like a configuration for mount locations across cloud provider
There's not going to be any reality to your role once you start either. Your devops role might suddenly become an IC role on their line of business app. It might become a support role. You might be asked to skill up on a stack you've never even considered without any notice or slack before you're expected to produce results in that stack.
The tech landscape changes too often these days. I have a friend of mine who was hired for his Magento experience but soon found himself learning Java after Adobe bought Magento. The things department heads think they want are rarely what they still want by the time you get hired.
It's ridiculous but there really isn't anything anybody can do about it right now. Tech is eating its own tail, the pace is accelerating, soon there won't be any snake left.
This is patently false. If your role changes drastically, and you don't want to stay in it, quit. But it does a disservice to workers to say, "You're trapped and this is just the way it is". It is not "just the way it is" unless you tolerate it.
Always Be Interviewing. Always Be Ready To Walk. You are a mercenary, and your job is to command top dollar for your skills while building your network so you always have your next opportunity at the ready.
The team/manager probably doesn't know what they want or does and has the budget for only a single hire. Their expectations are completely disconnected from industry/reality because they just don't know any better and probably lucked out getting into their position. They want a strong developer because the team needs one (or several), but they desperately need ops to keep their project afloat. So, sure, OP has all the right ops keywords on their resume, but can they also be that developer we really need?!
I butted heads constantly with my manager who had that mentality because of things like that. We rejected great DevOps candidates because we wanted them to be developers and good developers because we wanted them to know ops.
The teamsters are a trade union.
I don't write code with a gun to my head, clock ticking next to $100k in a suitcase, swordfish style, and not about to start now. ;-)
Moral: Take interview training and make yourself available to do interviews, especially if you're in a relatively niche role. Make this attitude the norm in the industry.
Nothing whatsoever.
It really feels like there’s no connection to the reality of the role whatsoever anymore.
You got that right. Basically what's going on here is that the people running these interviews don't really know what they're doing -- and are secretly terrified of being "found out". So in a desperate attempt to compensate, they lunge for whatever technique they vaguely heard about being used at, you know, "leet" companies. And (as others have pointed out) at (apparently) very little cost or risk to themselves.[1]
Because that way... even though basically don't know why they're asking that question of you... and not only that, most likely would not be able to answer an analogously difficult (and unrelated to their own problem domain) question themselves! --
if you do manage to make it through, they'll at least know that you're "leet".
[1] Except of course, the opportunity cost of false negatives, and the reputation risk (and damage to recruitment prospects) that inevitably ensues once your company becomes widely known for mindlessly cargo-culting outdated and discredited interview techniques. Which they would know about, if they knew that they were doing. But then again, apparently they don't.
If companies don't have strong enough incentives to change their interview processes, doesn't that kind of imply that it is working for them, at least to a certain extent? At least it doesn't seem to be the case that they are hiring so many bad candidates they are forced to change.
> I'm not trying to defend all current interviewing practices, but Leetcode-style interviews became popular because they're designed to find people who are good at programming without requiring overly specialized knowledge about specific tools that are popular right now but may not be in a few years. The idea is to find someone who can reliably learn what they need to know for the job rather than finding someone who's an expert in X and Y which you currently happen to be using.
I use synthetic / algorithmic questions for this purpose. I don't need tool specialists, I need people with wider skills.
I also like concocting fizzbuzz variants as a high pass filter. Some people fail them...
So I'm not surprised anymore.
* Does the interviewee have a basic understanding of what we do? * Does it seem as though they will fit well here? * How do they deal with failure? * Are they excited about working for us?
and the one thing that seems to be most important:
* If they don't know how to do something are they willing to put in the time to learn?
It seems to have served us well. I am always surprised when I read stories like this where candidates are turned away for not being able to build a quad-tree on a white board in 5 minuets whilst hanging upside down by their toes and repeating the alphabet backwards...
Do you know how to write SQL queries? Have you ever mapped an ORM to a data store? What about rate limiting an API? Ever had to setup Single Sign On? Have you ever scaled anything from 100 users to 100,000 users? How do you handle job running? Concurrency? How would you debug your server locking up due to 100% CPU usage? Ever configured a dev environment on local with xdebug? Breakpoints? Command line tools? Ever normalized data between multiple third party APIs? What does normalization even mean to you? What about unit testing? Mocking data pre-test and cleaning up after test? How do you make it fast? What's the autoloader? How do you properly setup composer? Tabs or spaces? Why? Windows, OSX, or Linux? Why? Favorite IDE? Why? Are their opinions so strong they can't work with others? On and on it goes.
In the past 6 months at my job I've dealt with all of the above issues and more. I would still probably fail most of the exercise in your Github repo. Mainly because all of them are irrelevant to the actual work and honestly I can't be bothered to sit here and memorize various math algorithms that have quite literally nothing to do with the work.
PHP comes with with 13 different sort functions. None of them test you for how to write a bubble sort. I'd rather a candidate know when to use one over another instead of knowing how to write just one or two of them from scratch. Here's the thing though. I've written bubble sorts before. In college. When learning C & Java. In no way shape or form do I ever practice them or actually remember them at a moments notice. I look them up like a normal person if I ever actually need them.
When I interview a candidate I don't use any of these nonsense exercises. I have one small code exercise at the end which makes use of recursion. The rest of the interview is an open ended series of questions on systems and just basic "shooting the shit" style questions. The nuance of how you answer the questions tells me everything I need to know about a candidate.
On the flip side if I'm the candidate and someone asks me one of these I know they actually suck at interviews and my immediate instinct is to walk out. I try anyway to be polite and not burn any bridges and most of the time I'll come up with a correct solution but again my initial instinct is "I don't actually want to work here anymore".
So what do you do instead? Be creative.
Have fill-in exercises. Write an abstract class and an interface and have them build you a class that implements both.
Write some code with obvious mistakes. Have them fix it.
Build a full from scratch login system. Have a user already in the database with a hash already there. Make them fill in the authenticate function. Watch them not use password_hash and ask them why. Make them use Google.
No exercise should take more then 15 minutes. The majority of the interview should be you digging deeper into their previous roles. Have them tell you success stories or cool hacks they've put together. You'll learn way more about them both as a person and as a developer.
A portion of those people who were filtered out because they get nervous during interviews or got stumped by a riddle or esoteric question, could have been entirely qualified to be a good or great engineer.
Big tech companies should be designing interviews that don't find just people who are awesome at interviews and riddles.
In my experience, a company using LC usually means that, if hired, 95% of your coworkers will be young 20-somethings fresh out of college. Generally, that is who has the time and lack of self-worth to put themselves through this process.
But the often-mocked LC interview wasn't as prevalent as I thought it would be. As a senior+ engineer I generally saw some system design whiteboarding and some take-homes, with a lot of engineer-run companies proudly proclaiming during the interview process that they don't interview that way. So I get it, LC is the standardized test for college grads entering the real world who have no professional experience to demonstrate. And that's fair, you gotta vet junior engineers somehow, I guess.
The one company that asked me straight-from-the-book LC-style puzzlers was an established SF-based startup that opened an office locally. Luckily it was a puzzler I had already watched someone solve on youtube and had memorized so thanks for testing nothing about my technical ability I guess.