How do your candidates normally work around this? Does everyone just talk about hobby or open-source systems they've worked on?
I would hate to be a stakeholder in a company whose value depends on employees not talking about and reusing the incremental skills and knowledge gained while working there.
I don't think knowledge on any particular api is particularly useful compare to the more generic "can you read docs and use apis" skill, which is directly applicable from something like high level hard ware work making an ftdi USB chip go brrr to making dynamoDB work nicely, but I definitely didn't know anything about dynamodb before doing a server/web job
Async distributed systems and FPGA vhdl have some overlap in how you build things, but doing an interview based on my old FPGA skills likely won't show much of my servers and databases systems design skills
If platform X is something public, they could talk about the platform. But, talking about any project specifics, implementation details, etc would be a felony.
Sure, folks will ask you to drill down here and there for more detailed info, but if someone pokes on something where the details really _ARE_ proprietary, it's fine to just say that and offer to drill down somewhere else.
Maybe we're just on different wavelengths, but literally every single thing I work on that would be a good candidate for this sort of interview is extremely proprietary...and I'm just working on regular ol' backend services at [generic big tech].
You wouldn't be able to build anything, as all your knowledge is NDAed!
Proprietary systems are company IP. Talking about the design of the system is protected by the NDA, which the exercise described above is asking the candidate to do.
You hire for the skills to develop your own proprietary system. The employer doesn't own the skill. The employer owns anything those skills produced for the employer.
You wouldn't want your employees giving the recipe to your secret sauce away to your competitor, right? That's what everyone is talking about.
[Client]->[Cloudfront]->[Load balancer]->[ECS service in a VPC]->[RDS/other internal systems]
* Replace the above with any other cloud provider, or just commercial/open source tools that do the same job running in a VPS or whatever *
I can't see how any of this could be claimed to be protected by an NDA or similar, as this is just a highly standard architecture.
If you work on a proprietary system that has a specific architecture just don't mention that of course. I think the above allows us to discuss things pretty well, talk about introducing caching, why each layer is there, TLS termination, authentication/authorization concerns and implementation, secrets management...
Oh OK! Then I guess a good engineer should be able to make it through a standard systems design interview process, and shouldn't complain about it!
I am glad you agree that it is totally fine to ask engineers these types of questions, and there is no problems with NDAs that will get in the way of them demonstrating basic skills!
That was my point entirely.
Yes, I totally agree that an engineer should be able to go through a tech interview.
And the engineers who are complaining about the fact that they have an NDA are wrong, and they should stop complaining.
If the NDA is so significant that it prevents you from doing an interview, then you are worthless as an employee anyway. But really that should never happen.
They can have used their skills to come up with a recipe that tastes the same because the requirements led to that outcome, or they simply copied your recipe.
If somebody from OpenAI goes to a competitor and creates something like ChatGPT, where is the line crossed that shows that company IP is transferred? The employee knows how to create the system because he knows how ChatGPT works. If he recreates a similar system but doesn't say that it is a copy of ChatGPT, is that applied skill or is that a violation of the NDA?
If it is a violation, how could he forget the structure of ChatGPT to genuinely come up with a new structure?
If I were asked a question like this I’d wonder if I were being tested to see how loose-lipped I am with sensitive information.
The first is generic (unless you happen to be a current Twitter employee). No NDA impact. The second is asking you to reveal details of a proprietary system covered by NDA (or if you’re a Fed contractor, covered by various classification laws).
In practice I can't imagine the spirit of any law would be violated by doing an architecture diagram, but it seems likely (to me, at least) the word of the law would be violated.
Sure, every company is going to have their 'special sauce' components, that ARE proprietary, but nobody's going to expect you to unpack those.
He worked for their competitor, but not in the capacity the interviewer was trying to delve into, so in addition to scummy, it was pointless and annoying.
At some point my friend cut the interview off.
His interviews are humorous. Those leading the interviews can't tell him what he'll be working on, and he can't tell them what he's done in the past.
The questions tend to be very theoretical, instead.
(Don't kill me, you all. I'm joking).
There is term called prior art an for me if you write bunch of props to database like text/numbers it is basically done in every other system.
Unless you build novel db system of course.
In theory yes, in practice ... if that's true, why would anyone ever hire you for your experience? You wouldn't be allowed to use it.
But isn't that exactly why you as an interviewer would pose a problem, set up the context that is potentially similar to the work your project would require, and see how the interviewee navigates that?
And I sure would hope they bring all their experience to bear! Just because they learned about CDNs at their previous job and “everything is proprietary”, I don’t want them to suddenly forget how CDNs work.
Very little is actually unique between software businesses. We’re mostly just doing data bureaucracy.
It doesn't necessarily have to be something that you're working on. It just has to be something that you understand well enough to describe the design of and how things are inter-connected.
So you're penalizing those under NDA and unecessarily limiting your candidate pool.
So it's really not a big ask to navigate their own NDA and come up with something interesting to talk about at an interview.
Besides which there’s a way more interesting variation on the question, which is “how would you design our product?” Got that one in my last big tech interview and quite enjoyed it.
He's not saying they ask you to give a full overview of the way Apple architects their systems, he's saying they ask you to explain how a basic Kubernetes deployment might look, or how you might go about setting up some kind of backup system, or literally any of a million different technical things that you might have knowledge of and interest in.
How could things like this possibly be under NDA? If you weren't able to use any of this knowledge, you would be completely crippled in your work.
Sure sounds like a good way to perform recon on a target company.
Just as an example, I do bioinformatics work, and "technical system" for someone I'm interviewing (as I understand your question) might encompass any of:
- a workflow manager: how it works under the hood, common patterns for modularity/reuse or configuration
- a specific pipeline: the stages of cleaning and quantifying raw data, the rationale for various stages and the tools chosen to execute them (including how to benchmark various methods against each other)
- a specific third-party tool: theoretical details of the algorithm, practical considerations for when to apply one over another
- familiarity with common general-purpose packages or APIs (e.g. pandas, numpy, scikit-learn)
- QC: common metrics for QC'ing various experimental data, how to decide if data is "good enough", how to troubleshoot biological vs technical (lab) vs technical (computational) sources of error
- biology: technical details of an experiment, the technologies generating the data, or the underlying biology
- data analysis: how to choose and make relevant figures, any of many data-science/ML topics (e.g. clustering), connecting data to relevant domain questions
- devops: data storage/management on HPC or cloud, HPC job schedulers, any of many cloud topics (e.g. IAC, setting up a database or cloud execution of a pipeline)
I'm more curious about how you guide a candidate towards selecting a system that gives you the most relevant perspective on their thinking and skillset - do you provide any suggestions, or are there typically pretty evident choices based on the role?
If I were in your position, one of understanding many complex systems, I would advise you to pick where you felt both strong and the panel was likely to have some entry point into.
Someone who answers, "I can't draw anything for you because everything I know about is somebody's private IP," isn't going to get the job. At least, not if I'm doing the interviewing.
You must know something about something else. Right?
I understand that you see your past work as “proprietary”, but in the real world the vast, vast majority of us do not work on systems shrouded in such secrecy. There’s nothing interesting or proprietary about CRUD infrastructure and I genuinely can’t think of a situation where you’d be exposing yourself to any real risk by explaining it to a new employer.
Yes, it is not that hard to keep the proprietary business logic separate from the system design.
I'm fact I'd even say that even if you haven't built a side project, I'd still use op's technique and ask the candidate to talk about and white board a system they would like to build and then dive deeper from there. Heck I'd even tell this to them before an interview!
I put in 60 hour weeks at my current job, except for a month before I change jobs, where I put in an additional 2 hours/night to prep for interviews, because that’s what will benefit me for every other company.
I care about my career and my family, so I don’t have the luxury of spending it on passion projects outside of work. I’d rather spend the precious little time I have left on my real hobbies and family/friends.
It doesn’t communicate that I’m a bad engineer. On the contrary it communicates that I give my all at work.
Having said that what you said about spending 60 hours at work tells others you will be a much better employee. Which is probably the signal they are looking for?
How many of those 60 are spent in meetings?
Superhuman working hours for someone with a family.
1. Despite the "let us work together" intent it is still an exam where you are penalized for taking too long or making a "wrong" assumption or missing a few bits,
2. Given above or not when confronted with a random question it can be easy to get very nervous unless you have built a similar system from 1 to 1b user scale or you have done tons of practise interviews with companies.
So apart from interviews being hard and what they are purely because of supply and demand I wanted to see if you can learn just as much from a more open book approach (with either a side project or candidate knowing the question ahead of time say the night before). It could even be more inclusive that way!