Caltech's CS interview prep course

It should be telling that there are a number of, usually for-profit, resources for prepping for just the interview process:

* leetcode

* Cracking the Coding Interview

* Elements of Programing Interviews

* Interview Kickstart, which costs like $5500, is a bootcamp just for passing interviews

* educative.io

* algoexpert.io

That being said, I might just play along with the DS / Algo interview game. Having an interview process that is artificially arduous and way outside the expectations of the job duties means that the job market will stay artificially inflated on the demand side. Hiring leetcode monkeys who can barely deploy and manage k8s cluster on AWS means the demand for my skills will be more valuable than if the interview process actually worked. I can complain all day to my bosses about how hard it is to hire (its not that hard) and meanwhile rip their HR budget to shreds when I threaten to leave unless I get paid more (I've done that before, I recently got a promotion and 50% raise on base alone).

So yeah, I'll gatekeep as long as others play too. I hate the game, but I will play it to my advantage.

Will you take the time and put in the work to pass one of these interviews? That measures perseverance and conscientiousness. Having put in the time, can you recall and adapt what you learned to perform complex tasks on a whiteboard in a stressful environment? That’s an intelligence test.

In combination, these criteria measure the famed, “smart and gets thing done” in a way that more practical lines of questioning might not. Or at least that’s the theory.

For the record, I’ve only interviewed at one FAANG company, got to the on-site interviews, did not get an offer (in case you think I’m secretly patting myself on the back here).

I'm always open to hearing about ideas to improve my interviewing, but I find that threads like this are repetitive and not really actionable.

One common misconception I keep seeing is the idea that the interview result revolves around solving the question. IMHO, that's a bad approach (which is unfortunately taken by many interviewers) and in fact it's an incomplete way of assessing candidates, given that most big tech companies have a list of competencies and/or values that are supposed to be evaluated. Big tech companies usually want interviewers to assess on multiple dimensions, ranging from communication quality to specialization fit.

As an interviewer, you're supposed to think of the question as a vehicle to go over these topics, not a pass/fail end goal in and of itself. In my interviews, I pace my hints with the goal of having the candidate complete the question, even if it means dictating answers to unblock them. There's nothing worse that sitting there wasting time feeling smug when the candidate gets stuck on something. Time is short, so it's much better to move on quickly to different topics than linger on something unproductive.

If you're on the interviewer side, my number one advice would be this: question the calibration of the question bank. It's usually a kitchen sink of everything the person writing it could think of, without regards for time. You might already feel that you don't have time to cover everything that the question is supposed to cover, let alone all the non technical criteria that leadership asks for. So instead, cover a few actual technical concepts (syntax, recursion, what have you) to rule out complete incompetence, and then take liberties to go into topics that do cover a wider variety of aspects. For example, discussions about testing, refactoring or debugging can all be productive ways to spend time in a coding interview.

</two-cents>

It's Google's interview format after all (actually Microsoft's, but operating systems are hard as well).

The great big sleight of hand trick behind all of this is that the entire industry has copied the interview process of a single, extremely large, company.

And Google makes $$$ writing software for *extremely complex graphing and search algorithms*.

It's a great interview process. For Google.

For everyone else, it's holding up a great big sign that says "I'm a moron and I copied my homework assignment from Wikipedia."

I wanted to ask the interviewer if they actually use these terms in meetings or used the algorithms, but I didn't. Since it was internal, I already knew the answers would be that they didn't, and I didn't want to appear confrontational.

I was internal, a high performer on my team at the time, and you're going to ask me this useless stuff?

Here is a fucked up environment.

Here are logins for each role.

Here is your team.

Here is your role.

Each of you has a chance at each role. There are three of you. So three challenges.

The environment is experiencing the following symptoms. Diagnose, report, fix.

---

Then throw them at some broken stack with admin rights and figure why its broken and fix it.

Create various stack models to throw at teams based on team skill-set.

Never hire individuals. Hire strong individuals who can work in a team.

Inspired by some realistic interviews that I did recently, I've been working on something[1] that aims to equip companies to hire engineers using realistic assessment methods. It scales like Hackerrank, but I think it's fairer.

[1] https://devscreen.io/

In the end, even if these coding challenges might not reflect the actual work you do, they might still be predictive of success in the job, as much as any other method can be when we look for unbiased markers.

This is similar to how the entrance exams to the IITs in india are pretty much completely discordant to what you actually end up having to know, the rigor and preparation you need to do ends up reasonably selecting for well performing students anyway. Not perfect though. Any system can be gamed but it works alright.

The very act of extrapolating how a person would perform based on a brief snapshot in time seems flawed. It is unreasonable to task the interviewer with making a good decision with barely any data.

I used to think that a well calibrated question bank or interview style could be a reliable signal. But the moment you ask something specific is the moment you don’t ask everything else. A poor performing but good candidate’s strengths could lie in the latter category and a well performing but bad candidate’s strengths could be in the former.

There needs to be a reliable and credible way to learn about a persons strengths and weaknesses based on their track record.

I’d weight credible reference checks, previous projects and credentials more than their actual interview performance.

The theory of interviewing should be to ask questions that are as predictive as possible e.g. success on the interview correlates as closely as possible with success on the job. The strengths needed will vary greatly by the role, company, and so on.

But what you tend to see is, everybody has a handful of "pet" questions that they ask over and over, without applying strict rationale for why its predictive to success.

Personally, any good interview question must both be iterative and highly correlated with actual work. Questions that have one "trick" and judging on that basis is a poor interview. Questions that focus on data structures or algorithms that aren't going to be relatively common are also poor. Questions which lack real world applications, also poor.

At least for me, I have a handful of questions, but one of my basic ones is to implement a certain data structure. And honestly, I care more about seeing the naive solution than the highly optimized one... because I care first and foremost that the candidate can code proficiently. The naive solution is super easy and requires little theoretical strength... but it becomes very obvious how proficient of a coder they are by seeing them write out the solution. How quickly can they write it? Do they make syntax errors? How long to grasp the intention of the question?

Some write the solution in 5 minutes, some take 40 minutes. That gap alone is probably the most strong predictor.

There is a more theoretically optimal solution, but I just followup at the end and we'll talk through what it could be. But if somebody can hammer out a naive solution with good code quality, quickly I weight that about 90%.

But my company also needs strong coders over particularly theoretical people. One of my best coworkers bombed on a tree interview questions I asked... but I could tell from all the supplemental details that he was very strong.

Yep,

Back in the 90s and early 2000s, the interview problems I got were often the actual problem I was meant to solve on the job. Which is to say that I don't think there's any barrier to coming up with questions like. Of course, if the job is as you describe, putting people through hoops is indeed a way to avoid actually telling people what they'll do.

I have. Many times.

If I had the choice between a Caltech grad who had all A's and no experience or a student from some random state school who had interned every summer and contributed to open source, I'd pick the latter. Many people are locked out from ever attending elite schools in the first place despite competence due to coming from poor backgrounds and underfunded highschool districts.

Is there more to you than your high school performance? Push back against this if so!

(Calling the interview Leetcode is misleading IME, since I've rarely been asked something that would be near the upper tiers of that site's difficulty).

Top-tier tech companies face a LOT more competitive pressure than top-tier universities; why should I trust the university selection process more than those corporate hiring ones?

Ask yourself why the rest of the industry isn't happy to simply hire every B student from CalTech/Stanford/MIT/Ivies without an interview?

I went to Caltech and would definitely not do this. The CS faculty was fantastic, but the undergraduate option has some problems:

- a wildly disproportionate share of cheating cases came from CS undergrads

- CS graduation requirements are very flexible, which is a mixed blessing. On the one hand, giving students more freedom is broadly a good thing, and many of them use that freedom to double major or go deep into specialized grad level work. On the other hand, some of them use it to construct schedules much easier than most other options would allow. Not all of them, but enough that "B's in caltech CS" is not as strong a signal as you might think.

Some signals I would consider stronger for Caltech students:

- CS/Math double majors. Or CS/[Any technical subject], for that matter.

- Summer research: someone paying five figures of opportunity cost to do something they think is interesting is probably not a $/effort optimizer.

- Grad-level CS coursework. CS 139 (Design and Analysis of Algorithms) in particular pulls no punches.

Why? This is not necessarily a good signal for competence on the job and also doesn't cover people who have lied or cheated.

Also, this wouldn't help create a less broken job market. You'd just be breaking it in a different way.

This was my first thought too. A course just for passing interviews seems odd. We had two small activities that were part of a business course (business administration minor), but after hours. One was a mock interview to give you feedback about professional appearance, not fidgeting, etc. The second was a mock business dinner to provide feedback about etiquette, customs, etc. These were valuable, but they focused on soft skills that aren't covered in the regular course content.

thanks, it means a lot you trust my ability so much :)

>All have PhDs and yet they’re all facing delays when landing a job. There is something definitely broken.

Everyone agrees it's the leetcode interview format. I agree to some extent, but I wouldn't throw it out completely. For me, I give four problems, and the candidate must solve two of the four with full internet access in an hour. It seriously reduces interview day variation for good candidates. If they can't do graphs but can do text formatting, they get a pass. I'm sure there's still some better way of doing this.

Welcome to "Education". We don't teach you life skills, but we will teach you stuff that you'll immediately forget and look up online the one time you need it in life (if at all).

How are you supposed to hire 50,000 people a year with a process like that? It seems unlikely there are much more than 100,000 people with significant OSS contributions in total. Published material isn’t going to be much more than that if it b even gets that high. And most people who are good at programming will have nothing for either. If you’re not going to spend at least ten, more likely 20 hours calling up references your going to be fishing from a pretty narrow pool there.

That is all commendable but none of that tells me if I can work with you. When I interview candidates I use whiteboarding as a collaborative opportunity to see how the interviewee thinks. I treat it as a similar experience when I am the one being interviewed, which is why I have never understood the hostility toward whiteboarding. How else in the limited window of time that is the interview can I learn as much about the people I would be working with?

Looking at past work penalizes people who primarily have experience in large systems where their work is NDA’d, impossible to understand without a loooot of background, or simply unshareable for any number of reasons. Also really bad for new entrants into the industry. They don’t have past work.

The take-home exam penalizes people who have lives.

The “do smol thing together so we can talk about it” interview penalizes people with performance anxiety.

There is no silver bullet. But if I’d have to choose, I’d pick the least bad typical interview approach that judges everyone based on their performance on a small job-relevant task. At least you avoid a lot of bias.

source: I’ve interviewed over 100 engineers over the past 18 months as we scaled the engineering org roughly 3x. I honestly don’t even look at candidates’ resumes anymore (this is after initial screens have been made) because it always leads to bias.

I'm in the same boat, yet most companies don't seem to care about OSS contributions, despite their rhetoric.

They want us to solve a few leetcode medium/hard questions within the span of an interview.

So you've done a bunch of stuff that's easy to game, that requires me to learn a bunch to even evaluate whether it's good or not, and you feel insulted by being asked to do some honest coding? I'd far rather hire a fresh grad who did decently on leetcode and was able to hold a vaguely normal conversation in the interview. In my experience that has a much higher rate of finding people who will contribute positively to actually getting the work done.

There's nothing wrong with the interview process, just a bunch of entitled prima donnas who are always going to think they're better than any standard process. You're better off without those anyway.

To your point about how you interview, unless there is an alternative I would never be able to apply to your company. I have no open source contributions or code samples to share and no interest in writing a book. I also have no interest in applying to places with take home exams where the company gets me to spend time interviewing without a commensurate investment of time on their part.

Where does that leave us? I guess it’s not ideal but, at least for me, the current process seems like it’s at least a local maximum.

Mate, wake up: ~billion dollar "interview prep" start-ups are a thing.

It doesn't tell you anything about the interviews, just about the achievement-driven personalities that the universities select for in their admissions process.

But looking at the bright sight, teaching young people how to “work the system” is not a bad thing. Again, we should fix interview. But in the meantime, a class specializes on “how to beat the man”, looks like a fun class where you can learn more than just CS.

I don't think most algorithm interviews correlate with what you actually do on the job though. There's a lot more communication (eg trying to figure out what the boss/client/pm really wants, explaining non-trivial concepts to impatient/bored colleagues, or trying to convince other people that your "crazy" idea is actually sane), and a lot less hard problems thrown "in your face" (eg. prove that there is no solution for a^n + b^n = c^n where n > 2), but a lot more hidden traps that don't show themselves until something blows up in production (eg. turns out a minor version difference of python triggered a serious outage).

While, if i was going for grad school instead, then i'd need a 4.0, start taking grad school classes, start talking with professors, try to get a job as a research assistant, try to find a professor that i could latch onto and start working with. And that was a path i had considered at some point, when i really did enjoy taking hard classes, going in deep, and pushing the edges of my learning.

[1] It's easier to pick a good teacher than to train one: Familiar and new results on the correlates of teacher effectiveness https://www.sciencedirect.com/science/article/abs/pii/S02727...

Now, those places were in now way "prestigious", or paying $250k starting salaries. Just your garden variety F500 companies / gov. agencies / non-tech companies.

Also, my experience with "traditional" engineering companies have been similar, 1/10th duration of tech interviews, some fundamental questions related to your trade, then mostly questions on personality, you experience, etc.

The closest I've been to tech, was finance. It's very much a predictable process - you know what kind of questions you'll be asked, what the next round will include, and all that...but still not anywhere near the rigor of prestigious or trendy tech firms.

Unlike doctors, lawyers, dentists, civil engineers...

So you have a well paying profession, hot demand, good working conditions, and no external framework to ensure a base level of competence. Of course people will try and game the system (wittingly or otherwise).

IMO this is a good thing. It allows people from all walks of life to get into the industry, and has helped it to grow more rapidly. But it does introduce a big issue, that its very difficult to know if someone is an expert, or just trying to fool you (and perhaps themselves) into thinking they know what they are talking about.

HN is a poor barometer for the broader tech industry - everything from hiring practices, tech choices, etc. HN has a tendency to make it sound like all the world acts and smells like a giant tech company or a startup. There are a TON of companies that are not like those.

And even as an L1, you can still apply at other jobs, because they will apply for the H1B for you and if successful, hire you then.

So yes between the waiting periods you do have to keep it up, but the barriers are not that high compared to citizens otherwise.

And this is my personal experience as a visa worker.

> it boggles my mind anyone would invest so much energy in getting a job at Amazon.

If they can sponsor a visa... just consider the sort of poverty that exists in much of the rest of the world.

I can't comment on the working conditions at Amazon but I have to say that those Indians must be highly qualified to have that job and should be paid fairly.

Nothing you describe suggests to me that it was the interviewing that was broken, it sounds like the employment was broken.

How far from this is the A level in Computer Science[1] which should take at minimum 20 weeks full time study for the target 16-18 age group? Or the EdX MicroMaster’s in Data Structures and Algorithms[2]? If any one of the FAANG companies said “Get this qualification and we guarantee you an interview” that would already be massive, never mind skipping one or two steps of the interview process.

[1] https://www.ocr.org.uk/Images/170844-specification-accredite...

The content of this A Level in Computer Science is divided into three components: • Computer systems component (01) contains the majority of the content of the specification and is assessed in a written paper recalling knowledge and understanding. • Algorithms and programming component (02) relates principally to problem solving skills needed by learners to apply the knowledge and understanding encountered in Component 01. • Programming project component (03 or 04) is a practical portfolio based assessment with a task that is chosen by the teacher or learner and is produced in an appropriate programming language of the learner’s or teacher’s choice.

[2] https://www.edx.org/micromasters/ucsandiegox-algorithms-and-...

1. A race to the bottom where licenses become easier and easier to get, until it becomes no more a demonstration of competence than drivers' licenses are to one's driving skills.

2. Protectionist policies make licenses mandated and difficult to obtain, then the talent pool and innovation moves to places that don't require that license.

Each state tends to have their own (often quite different) license requirements. They tend to be expensive to get and need to be renewed frequently (more $$$). There are usually other hoops you have to jump through to get both license and relicensed. I would never wish licensure on my own profession.

https://ncees.org/engineering/pe/software/

But nobody was interested.

https://ncees.org/ncees-discontinuing-pe-software-engineerin...

---

I've interviewed software developers who have on their resume "Oracle Java 8 certified", but aren't able to name three different spots where the static keyword appears in Java or the situations where they'd be used.

There are certainly many possible reasons why someone wouldn't retain that knowledge, but I've encountered far too many candidates claiming mid or senior level experience with "enhanced" resumes and certifications from everywhere that are not able to satisfactory do more than follow instructions (e.g. not able to determine steps needed to complete tasks nor able to solve the larger problems and break them down into tasks).

So, with the resume and certifications not things that I have confidence in, asking those questions to try to figure out if the person is able to approach solving a novel problem is what gets asked. If other companies do the same, then that candidate will likely see similar lines of questions in each place.

I'm not sure if licensing will work. We already have "licensing" via Microsoft/etc certificates. And my experience is that we test these people with certificates even harder because of the poor reputation of these certificates.

But people from those places still perform really poorly in interviews at about the same rate as everyone else. I wish I knew if it was a fluke that they bombed this time or that they passed the original.

Anyone can go online and learn enough to be useful at a high paying entry level job.

At some point there has to be a filter and that's what these interviews are for.

I am willing to bet that spending an hour every day on leetcode you can probably be good enough to pass interviews at most companies in a month or two (assuming you had exposure to DS and Algorithms in college, if not add more time but the general idea applies). Maybe not Google but I would bet you'd be good enough to see double digit increasing in your onsite -> offer conversion percentage. Whether that would be worth it is a different question, the answer for many people who already have good jobs is no.

I graduated with a physics degree and I still remember how steep the programming learning curve felt when I took my first programming in C course despite having done some python work in the past.

So I don't think the "most" qualifier is necessarily correct either.

What incentive do tech companies have in keeping salaries high?

(Just to conserve space there, I detached this comment from https://news.ycombinator.com/item?id=30492698 and marked it off topic.)