Hacker News Reader

Comment by kristopolous | Hacker News Reader

I look to see if I can trust them to not do dumb things. That's really it.

What's the ego, arrogance, competence, willingness to ask questions, interrogate things, redefine problems to be less work (or none at all).

Do you see what's not there? Programming language competency. Database knowledge. Things like that.

I've been hired as a Ruby programmer, I didn't know Ruby. I was hired for mainframes in the 90s - I had never seen a mainframe. I was told about 20 years ago to write a program in perl, my boss gave me a perl book with the assignment because I didn't know perl and I did it.

This is not a bad way to go about work. Every codebase is new. Every job has a steep learning curve. Every requirement is a conversation which may only be internal, but has to always happen. Every programmer you work with is someone whose preferences, strengths, styles, and weaknesses you need to have intuition on. People good at those things are what make good programmers.

That's why general competency, ability to learn, not to create a mess and not to waste time on stupid things are the only things I care about. That's really it.

Preexisting technical skills aren't as important as how dynamically you can apply new ones.

View on HN

I interview quite a lot, and one of the things I try to do in an interview is work through a problem that I almost certainly understand more deeply than any of the candidates (both because it's my niche and because I've worked through the exact problem with hundreds of candidates before). There's a certain minimum bar that I expect any educated and intelligent individual to achieve if they're a good match for the job, but then I take it really far beyond what I actually expect them to know, to the point where a lot of the details are unknown, and the details matter a lot in determining what kinds of solutions will be successful. Do they start BS'ing me? Do they confidently walk into really nasty parts of the problem without recognizing the pitfalls? Or do they tell me they're a little stuck? Do they ask which trade-offs to make? Do they ask if they can do some Googling for prior research on the problem? Do they explain what they would try and what pitfalls they would watch our for, or how they would validate their idea? Do they ask if they can make some simplifying assumptions for practical purposes? There's a lot of right and wrong answers but really I want to see their attitude and communication.

My first couple of hires before I did this were a brilliant computer scientist and a guy with a business degree and some coding background. I was surprised to see the guy with the business degree excel because we weren't doing many CS-hard problems, and an ability to manage time, talk about the problem, and navigate the project life cycle were far more important than classic algorithm knowledge. He needed to know how to code and debug, for sure, but like most software jobs, his soft skills were a force multiplier.

I'm pretty sure this has been done with me - I was given a problem and told to come up with something like an O(n log n) solution, and I honestly couldn't do it. I tried a few different approaches but I could think of really bad pathological cases for all of them. The interviewer gave me a few suggestions which I explored and I kept exploring them and then admitted I didn't see a way forward. I thought I was bombing and then he told me I did really well. I looked up the problem and found out he had done Ph.D. research in that area, and there was no general O(n log n) solution - only solutions that could do that in most practical cases.

I would think that anyone hiring a Ph.D. in Computer Science would be looking for a different kind of output than the knowledge of actually doing software development and the associated project management/time management work. I would imagine that such people, if placed optimally, would be doing largely self-directed work with an expectation of solving a very complicated problem, with work output that looked more like a thesis than a program.

I further surmise that very few companies need these kinds of people. If your needs are people that can be managed to do basic programming tasks (my job as well), then that's what you hire. That's something like 99% of all programming work.

It's like hiring someone that understands the building of nuclear power plants, and being surprised that they don't understand basic home building codes.

The saddest part of it, for me, though, is that someone with a Ph.D. in Computer Science that is searching for a job in the non-research sector is probably having a really bad time. Research is happening, but I don't think it will jump in a very large way until there are a lot more dollars being pumped into research using public money.

Just to be clear, I neither have Ph.D. in CS myself nor am I looking for that in candidates. I agree with everything you say.

I think that would be kinder to the applicants if you told them that there were going to be things you didn't expect them to know required for a good solution and you didn't necessarily expect a good solution but were interested in how they approached it. You'd still be able to tell nearly as much by what questions they asked, and it would be less testing for "How well do they do under unrealistic/unrepresentatives kinds of pressure and in guessing what the boss actually wants when he isn't being honest about it."

One of the most unrealistic parts about a code interview is just how high-pressure it is, time sensitive in unrealistic ways, way more impact on your career than usual 5 minutes of coding, no peers to ask questions to, unclear if asking to google will be seen as a negative or not, etc. I think trying to reduce some of these unrealistic pressures, by being clear about what you're expecting (and being in this case up front that you _have_ intentionally given them something out of their domain knowledge) can probably give you a better evaluations. Some people do better under that kind of pressure than others, but if that kind of pressure is not actually representative of the actual job, it may not be what you want to be testing for.

Also set expectations for transparency and kindness (instead of bullshitting or preening) by modelling it yourself, right from the start.

I generally do tell them it's a complex problem and I'm looking more for how they approach it than any specific solution. I don't tell them everything I'm looking for because then they'll just pretend to be the way I want them to be. But I disagree that it's unrealistic. I've had far too many employees and coworkers who don't understand what is being asked of them or don't know what they're doing, and try to fake it as long as they can. And if I tell them they need to tell me if there's a problem, they almost always haven't until the pattern continues and becomes a worse problem. If the way they deal with pressure in an interview is to bullshit me, I need to know that long before they're under pressure at work.

I like this idea because it allows me to figure out if they have one of the most important skills: being able to say "I don't know" without wasting a lot of time.

IMO anyone else saying you should tell them the solutions might not have answers are dancing around needing to have that skill.

I guess if an applicant has done enough research/prep for technical coding interviews, they probably will come prepared realizing that not all questions may have answers or may be able to be answered without context they aren't expected to have.

I know I will after this discussion!

I know I would also get very thrown by a question without an answer if I wasn't expecting it, in a way I don't think I would be on the job. (On the job I know too well through experience that not all problems have good solutions, but in an interview question I maybe would have assumed there was an answer they expected me to get, before this exchange).

Of course, the end goal of interviewing for an employer is maximizing their chance of filling position with good people, which involves balancing false negatives and false positives -- and in fact a "false positive" (wrong hire) is probably _worse_ than a "false negative" (turning down someone that would have been great). So... filtering out some candidate that might have been good may be a neccessary sacrifice, not a flaw in the process, so long as you're doing even better at successfully filtering out those that wouldn't work out. Whatever works!

If I felt someone was intentionally giving me a "trick question" in an interview, I might conclude they weren't someone I wanted to work for anyway, so everybody wins I guess.

It doesn't even have to be a question for which there is no answer. IMO it works best with progressive questions where you can start with something straightforward that just validates basic knowledge they should have, and progresses to get increasingly difficult.

I used to ask about the layers in the network stack, and how you might troubleshoot it. I start with something as simple as a firewall blocking a port, or the server not listening on the port you expect. If they do that, great. Anyone who's had to troubleshoot distributed systems should have checked that stuff plenty. Then we try a protocol they might not have used. Most developers have a rough idea of UDP vs TCP, but they might be confidently wrong about some details or try to make stuff up and say "I'm pretty sure...". We go down a layer and try some scenarios where routing is messed up. I don't expect most devs to know about that, maybe someone with more sysadmin experience, but by now it's painfully obvious if they don't know what they're talking about. It happens to be pretty handy on my team, but if at this point someone says, "you know I really don't know a lot about networking", great! I'm glad you told me. Let's skip talking about voltage fluctuations in Cat5 cables and get back to software development.

Another good one is designing a data structure where there's a lot of different access patterns you might optimize for. Gives them a chance to demonstrate some simple coding and CS skills before it gets complex, but then there's lots of room to talk about the use cases. Once you're talking about use cases, are they willing to ask about the trade-offs? Priorities of various use-cases? etc.

Then problem is that getting to "I don't know" can take a LONG time for someone who's actually a good fit. A generalist (jack of all trades, willing to become a master in some that are used often in a local time domain) might actually know a lot of that information.

A good way of approaching this might be to provide problem feedback during the mock, stating that there's a given problem reported with the results or some other plausible complication and seeing how things go from there.

Also, the not knowing thing. I've hit that a few times myself, however I'll generally exhaust every reasonable option I have to look-up or reverse engineer the solution on my own first. Google / duckduckgo, stack-overflow, asking some other tech friends if they've encountered similar things before; if I worked somewhere with other technical users that I expect might have seen this they'd be just before the out of company friends on that list.

By the time I get to something I don't know, and can't find out on my own, it's actually a DIFFICULT or VERY domain specific problem. Generally then I go for adding diagnostic prints or using a debugger to trace the problem and find out where things are going wrong so I at least can narrow down causes/solutions. Even then, it's still taking a __LOT__ to get to I don't know.

Getting to an absolute "I don't know" does indeed take too long, but doesn't really do much for me as an interviewer anyway. My goal in these questions is to get to "I don't know, but I think... and I would verify by...", "I don't know, but one could try..." or "I don't know but I would Google for..." are all excellent places to end up. You're not BS'ing me, but you're not stumped either. You know sensible next steps. We all have to look stuff up regularly or test the unknown.

Can be a problem trying to out-generalist a generalist, but if you pick something in your niche (I ask them lots of other questions relating to their background, so this question doesn't have to cover the exact job requirements) and practice the problem with dozens of interviewers, it'll be pretty hard to have someone who can genuinely out-think you in the depth of that problem. And missing out on the "I don't know" is a small price to pay to find a candidate that does that well on the problem!

" I looked up the problem and found out he had done Ph.D. research in that area, and there was no general O(n log n) solution - only solutions that could do that in most practical cases."

The Kobayashi Maru of programming interview questions!

If I smell any bs then I immediately change tactics and make sure I put them deep over their head. For example, "what are the strengths and weaknesses of classifying grammars via the Chomsky hierarchy?"

There's 3 ways to answer these kinds of questions and it exposes you as either a genius (or in this case a linguist), a non-genius (real geniuses are as common as 7.5 ft tall people) or a bullshitter trying to look like a genius.*

I've never met an actual genius in an interview so let me address the other two. That third group is a nonstarter. I've known the person for 10 minutes and they're already trying to deceive me. How the hell am I going to build a product with them?

Kim Scott, some SV big wig wrote an entire book on this called Radical Candor. Essentially it's "don't bullshit people (and know where your taking the relationship when you use that rule)"

I strongly believe companies work best with the smallest bullshit quotient possible. They may sell more or be more profitable with certain flavors of bullshit, but things are easier and more stuff gets done without it.

At the end I would have rather actually made something than duped people with sham products sold on dreams

---

* I actually can't answer that question but I could probably identify the neighborhood of a real answer (...something something limitations of the von nuemann model restricting the constructions of parsing systems something something...). The point isn't to answer it, although that would be quite impressive; it's to see how far someone digs into their own BS.

I'm curious about the question here if anyone wants to chime in (this is just an area in CS that I'm interested in but don't have exhaustive knowledge of or anything).

> what are the strengths and weaknesses of classifying grammars via the Chomsky hierarchy?

Are there alternative classification schemes for grammars? I haven't come across any personally—not for formal languages anyway.

Or was something else meant by the question?

For Chomsky, I would hazard finite state automata, BNF systems, and Turing machines. I might then mumble about number of tapes and time to recognize, and then stop.

Type 0, Type 1, Type 2 and Type 3 (aka: Type K grammars).

Type 0 is an unrestricted grammar.

Type 1 is a context sensitive grammar.

Type 2 is a context free grammar.

Type 3 is a regular grammar.

For our (CS) purposes we typically use Type 2 and 3.

For example, "what are the strengths and weaknesses of classifying grammars via the Chomsky hierarchy?"

What's the point of grandstanding like this? To make yourself feel smarter than they are?

My way of dealing with bullshitters (who in general are surprisingly eager to reveal themselves as such -- there's no need to lay special traps for them) is simply to say "Hmm - okay" and politely end the interview at the nearest reasonable opportunity.

It's a 1-on-1 interview (not in front of others). Ultimately the point is to help the person.

When interviewing and I didn't get a job I always desperately wanted to know how to improve things next time and nobody would tell me. Instead there was this little canned BS email from HR "after much consideration blah blah blah" stop that ... tell why!

This goes one step beyond telling. Show people why. Show them what they are doing, when they are doing it, how you can see it, why it's bad, why it hurts them and help them not do it in the future.

Give them an offer to correct it after you've made it clear (this is like throwing an exception in programming versus silent failure). Reset the clock and try again if you want. Maybe they're actually great and you read them wrong (they might be a "good program" with your bad input). The interviewer isn't some flawless executioner.

Most interviewers don't act like characters out of an Ayn Rand book. At the end of the day they care about others and know they have bills to pay. Doing this shows "this is why this job can't happen". Try to be their best interview for a job they didn't get.

The candidate doesn't walk out the door with ambiguity or false pretense or the illusion of doing well. Nor do the decisions come across as arbitrary and indiscriminate. They know exactly what action they need to take for their next opportunity.

If they don't get the job on offer, the best you can do for them is help make sure they can get the next one.

Fair enough - thanks for clarifying.

Sadly I’m yet to meet an employer who thinks like this. Especially in Australia. I’m a C# developer not because I was born with C# DNA but because to use any other language in suddenly looking at junior wages. Even though I could do a great job in any programmming language. It’s insane.

I've worked at 4 software companies in Australia so far and none have been concerned about which programming languages I know (admittedly not for super high wages, but still...)

So how did they do the code test? Did they say you can do it in any language?

I interviewed for atlassian a few years ago. They gave me a tricky Java test, the kind that you’d only pass if you’ve been battling Java for years and know the gotchas. So that was a fail. I could have studied Java harder, but I’ve only got so much time and so many opportunities.

1 don't remember, 1 any (but need SQL), 1 Java (Atlassian, actually, but only the easy screen - may have been the particular division here), 1 no test because they knew me.

out of curiosity how high / low relatively are the aussie wages compared to us ones ?

They are lower, but better than continental Europe.

I here that SF people are getting $130k out of uni. That would be a rare high salary here (180k Aud).

Probably a manager of 100 people or a good cider working in trading, or a smart employer who doesn’t want to play the market rates game.

Salary is total comp here pretty much.

I wish everyone thought that way but they don't. In today's rapid paced world if you don't know Rails (not even Ruby) but Rails, for example, and we're using it - then you ain't in... and here write this 8 hour code test so we can subjectively judge it because grading code tests to see if you don't put logic in your controllers make our egos hum.

Maybe nitpicking certain micro-design choices isn't always helpful, and maybe it doesn't need to be a task requiring 8 hours, but I believe a "work sample" of actual code is the only technique that has any research backing it as a good way to evaluate potential hires.

If you're recruiting with a specific company and specific team that you love, sure. But I can't imagine committing multiple hours of my time to each random company's take-homes. As a rule I don't really do any take-homes - the couple I've done haven't been very useful in terms of my own time investment into the problems themselves, and the companies have not been worth it to go above and beyond.

This is my exact issue with take-home tasks: I can get to an onsite or a rejection with most other companies after a 15 minute recruiter call and a 45-60 minute technical phone screen. So, why do I want to do a 2+ hour homework assignment? Most of the time, there simply is no compelling reason to do so.

I greatly prefer take-homes, because I feel they show off the quality of what I can do under actual coding conditions (rather than the stress of live coding or the BS of whiteboard coding) but it does get a bit tiresome. Especially because it is rarely a 2 hour assignment, they can be 8-16 easy.

Recently I've had three or four interviews that didn't quite get to the job offer stage, but they did both include over 30 hours of work + interview steps. I'm sorry, but a full week's of unpaid work that I have to fit around my actual day job is just not practical, I know you want to make a good decision, but really...????

The best I've seen recently is a company that paid an hourly wage for 2 days worth of take-homes. Not consulting wages, but a good wage. It gives them incentives not to invite everyone in the world to do their test, and gives the interviewee something for their time.

> Recently I've had three or four interviews that didn't quite get to the job offer stage, but they did both include over 30 hours of work + interview steps.

This lines up exactly with my experience. 3-4 interviews, each requiring ~30 hours of technical tests, and interviewing. Absolutely ridiculous.

I wish there was like one long interview process I could go through - I do a bunch of technical work, I answer some standard questions on video, a bunch of companies review it all, and the ones that are interested schedule a 1-2 hour personal conversation to make sure we are a good fit.

I suppose that is what triple-byte is trying to do.

But really, we're all at-will employment, let's just work together for 3 months or so, see if you like my skills and I like your culture and if not just part ways - I'd even be happy to work at a wage cut for the first 3 months. But having a full-time job interviewing while working a full-time job is exhausting, esp when after 30+ (!!!) hours of interviews and conversations, you can be cut at the final stage without a single word about what went wrong.

Regardless of what Triplebyte is trying to do, they don’t actually work the way you suggest. You take a short, multiple choice quiz, then do a coding challenge and an interview. I believe the time investment is around 4 hours.

What you get is actual, real feedback on your performance, and you skip straight to onsites with companies they match you with. The feedback is valuable, but I don’t know how their matching process works at all.

What it _doesn’t_ do is “turn an O(n) process into an O(1) process,” as their FAQ suggests. All it saves you is 1 phone screen per company they match you with. If a typical phone screen is 45-60 minutes, and the time you spend on their process is 4 hours, you need to interview with 4-6 of their clients to make it worthwhile in terms of time savings.

IMO, don’t try Triplebyte to save yourself time. Do it for the interview feedback, because that’s the best place I know where you can get real feedback for free.

Ture, but for some companies the "technical phone screen" is multiple interviews + a take home. Etc.

If I get to skip 10+ hours of interviews and coding work, that's a significant time savings

8 hour code test - you are an optimist! :D Recent trend in Silicon Valley is 1 week to 1 month (MVP) coding test in a hot area (e.g. Deep Learning), preferably coming with unique world-class solutions during that time :DDD And you get a phone interview only when you went to a Top 10 school.

In this market? I'd laugh in the interviewer's face. Like hell I'm doing a month of unpaid work for your "prestigious" opportunity.

Yes, if you are competing for high-end salaries/share %.

It's another way for them to age discriminate.

I guess, I just call it demanding I work for free, and that ain't gonna happen.

well then fuck those companies. The people you work with will be the people who made it through that kind of filter.

Their other processes are probably equally terrible and their codebase is likely a nightmare. Make your money somewhere else. Really.

This has become more worrisome of late since the number of workable toolchains seems to be increasing. The fraction of jobs in this town I could be eligible for (without a lot of tapdancing) is much smaller than it used to be, despite the fact that we aren't really doing all that much that's different than ten years ago.

Couldn't agree more. An ability to learn quickly, to self-teach, to solve problems is vastly more important than any degree of exposure to a specific piece of technology. These characteristics are as much about your skill level as they are about your attitude and approach to software engineering, as well.

> Do you see what's not there? Programming language competency. Database knowledge. Things like that.

I hear this a lot, but I'm not sure I agree 100%.

I want to work with people who can write idiomatic code and are able to use a database efficiently (or who are at least willing to learn these things).

I prefer working with idiomatic code. On the other hand, we hired a guy who didn't write idiomatic code but who was a brilliant engineer in general and he was overall a great hire even if I had to bring him up to speed on a couple style points. Overall, I found his general engineering chops dramatically outweighed the minor annoyance of tidying up style foibles.

Some small counterpoints on idiomatic code:

1. I've worked with people with years of experience in a language who can't (well, really, "won't") write idiomatic code in that language. This has—in my experience—typically come down to overconfidence/arrogance built on top of their experience whereby they've spent enough time with the language to develop their own "better" style.

2. Learning quickly on-the-job typically involved picking up idiomatic style first, before you really gain a deep understanding of the underlying workings. Ideally a good engineer should gain both, but at least initially, I haven't found the former to be a problem for "good" engineers starting out on something new.

3. For some language ecosystems, what's idiomatic changes over time. Flexibility and adaptibility is key to being a good engineer, but if someone has a lot of experience but isn't so adaptible (and/or hasn't brushed up on modern techniques in their ecosystem), this can also lead to non-idiomatic style.

I get that some of the things above should be red flags "in an of themselves" when hiring, but I'm just pointing out that a focus on past familiarity with tech can cloud things somewhat, and shouldn't be implicitly prioritised.

Sometimes idiomatic code results in code which is hard to understand, as exemplified by the C idiom `while (s++ = t++);' for copying a string from t to s. It's always good to be able to read idiomatic code, however.

It sounds a bit like you are primarily selecting for high IQ, with a side of good attitude. I tend to agree with that.

My experience has been that interviewers like you are rare. I may have reflected on my observations incorrectly, but it seems to me that depending on a newcomer to quickly learn a language and adapt is seen as a high-risk investment. I consider myself an adaptable person, and while I was able to make this impression upon prospective employers in my early days, they would always weigh this against the cost of training me for "longer" than the average employee.

In a sense, they were right. I have noticed myself increasingly comfortable with new codebases whenever I change jobs, so experience definitely counts for something.

Yeah, too often I've had interviewers who were far too eager to go and go all the way down to the minor corner cases of a problem (even if I try to beg off of them) and in the end I have no time to ask interesting questions. Questions that might reveal more about my skills than the last two corner cases.

In your opinion are there any common signs you see that indicate someone is good at dynamically applying new skills? or even better is there any signs that clearly idcated they aren;t good?

you talked about paying attention to their attitude but I mean more specific things like, if they said they use Emacs and write bash scripts would you take that as arrogance or that they are competent?

That's what happened at my first job - told to get the book on Fortran from company library and learn it.

To be fair, I found Fortran 77 to have the easiest learning curve of any language I've had to pick up. It only required about 2-3 hours of reading, plus a little more Googling to understand the challenges of having it interact with C++ code.

> ability to learn, not to create a mess and not to waste time on stupid things

People do stupid things and create a mess, while they learn.

They are learning at a new job anyway: the code base, tech stack, and social constructs that contributed to these. Learning the syntax and idioms of a new language is the smallest thing in most cases. Unless the language is particularly esoteric I submit that worrying about the "mess" made by learning it is hardly worth considering compared to those other things.

How do you check for those things in an interview? I'm not suggesting you can't, I'm genuinely asking.

You talk with them. You've probably worked with enough people that you can sniff out the fakes

Reply on news.ycombinator.com