I hope it ushers in a new era of non-leetcode style interviews.
But I suspect what it will actually do is usher in an era of in-person only interviews or having to use that same crap spyware that schools use to lock down your computer by essentially rooting it.
The most extreme one I’m aware of is “Lockdown Browser” and the student who lives with me has effectively rendered it useless. Students use MST with DisplayPort to mirror a monitor in a way such that the OS cannot see that two monitors exist. (I don’t think there’s a windows API for LDB to see this? Could be wrong)
Anyways, one student faces the other, each looking at a mirrored display, one student out of view of the webcam. Then a microphone with a hardwired switch soldered in which, again, doesn’t alert the OS that a microphone has been disconnected, is switched off.
Then student 2 can freely speak at the student taking the test and announce all the answers. The test taker is able to keep their eyes on the monitor at all times (so eye tracking won’t show anything weird).
If I were black-hat enough to lead a product like lockdownbrowser, I would beat this technique by looking for electrical hum frequency signatures in the audio feed which can pinpoint what time the audio was recorded due to fluctuations in the grid electricity. https://en.m.wikipedia.org/wiki/Electrical_network_frequency...
If there is zero audio waveform, that would be a flag for review. If the cheater attempts to loop pre-recorded audio, it will be noticeable that the ~60hz frequency signature is “wrong” for the time that the test was being taken.
This would then in turn be defeated by the cheating student running a remote microphone to a quiet room which can be switched to.
I’ve put a lot of thought into this and no matter what, the cheater eventually wins given infinite cat-and-mouse iterations. But most would be caught somewhere along the way.
If people complain it's impossible to fire anyone, that's the issue. The solution isn't to implement an insane anti-cheat hiring pipeline that will drive away competent people.
To do what? Why would the cost of hiring scale with the salary of the position?
I have often seen this problem explained (for example by Paul Graham) that a bad employee is a negative value, they make bad commits and stupid decisions that net out as a massive negative for the company. But it seems very counterproductive to try to solve this problem at the selection moment using stupid proxies such as leetcode memorization, instead of during the trial period, when the employee is, you know, interacting with your company.
I don't see anyone explaining how that latter number gets so high. I assume most of the cost is the time of multiple interviewers across multiple candidates, but that still seems outrageous to me. Even 1x the employee's salary would be 4 similarly-paid colleagues doing nothing except interviewing candidates 40 hours a week for three months, which I've never seen happen. Even the most grueling interview process I've been a part of has been more like 2 hours a week for a couple months.
I imagine it includes onboarding.
• Recruiter/sourcer fees. These are typically a fraction of the salary, so that right there is a large chunk of it.
• The cost of all the interviews needed to locate enough candidates that one accepts. If you interview 60 people in order to make one hire, and conservatively assume an interview takes one hour for doing it and one hour for prep+writeup+hiring manager/committee analysis, then that's 120 hours of skilled labor.
• Hiring bonuses.
• Relocation fees.
• Travel costs for all the people you interviewed on-site.
• On-boarding cost (HR, legal, IT setup, possibly desk provisioning and equipment purchase).
• For some types of employees, time spent in negotiation, meets and greets etc.
• Cost of the ATS.
etc. There's a lot that goes into hiring someone!
What throws me is the claim that this is possibly around the average.
There are efficient companies out that there that can pick through the resumes in a few hours, arrange a few interviews, and have somebody hired in a week or two.
So if there are efficient companies out there dragging down the average to a year, does that mean there outliers out there that are spending 2 years, 3 years , or more worth of salary to fill a position?
If the company wants to make hiring incredibly difficult (too many companies do), I can see it costing that much.
So stop doing that.
Having been in many startups doing a lot of hiring, if each hire cost a year of salary we'd have burned through out funding many times over and yet we didn't. So it doesn't have to cost that much. Keep it simple, hire fast.
It's the lazy cheating that's really disappointing.
We have organised crime targetting our events, they have done things like infiltrate IT departments and trojen horse PCs. For high-stakes, we advise clients that in person inviligation in combination with LDBs is the safest way with a freshly provisioned SOE.
If you want to make this your job, work remotely and like C#, u can find our careers page with this hint: 3F27483F97C94ECF0C8F11148FBBD048DFFCDECBE5C62FA23076297AE804F6C6 Send us an unsolicated resume and say ur from HN LDB
The person you found probably thought that SHA256 is enough to obfuscate customer names and didn't know that rainbow tables exists.
I have a cheap $50 capture card that is untraceable. My $75 HDMI mux has hdcp stripping and EDID pass through. I use these for streaming video games and this is a pretty standard set of kit.
The only way to discover this is hoping that custom control commands are passed through to the monitor. OEMs use these to give you access to the OSD options from the operating system, such as a crosshair at the center of the screen for gamers, on when their game is running but off when it exits.
> This would then in turn be defeated by the cheating student running a remote microphone to a quiet room which can be switched to.
Honestly, most of the noise comes from the pickup. Using a simple switch to ground will still allow that hum through. You can also just loop a power cord around the mic line a few times and "actively" induce that noise.
My headset makes a terrible warbling modulated at a frequency of 1/120th hz from the bad isolation in my KVM's power supply.
No, a far lower tech option exists: more and more Console gaming targeted headsets like those from Steelseries include an aux in, which can easily be used for the snitch. Egg cartons for sound dampening and impromptu vocal booth and you're well on your way to defeating any technological measure.
Bonus points the next time a candidate (correctly) uses ChatGPT or Copilot. Let the machines do what the machines do well (grinding leetcode), good riddance.
Probably easier for most to list what it doesn't look like.
I'm interested in both.
For example: What’s the method to select a random item from a range in Ruby? (ChatGPT used to get this wrong.) I don’t mean to say that I give trivia questions in interview, but if the need to know this came up during an interview or code pairing, having the candidate know when a chatbot response was invalid (and where to look for a correct answer) is a good sign.
I’m also open to a candidate stubbing out an idea with a chatbot/copilot and then checking the solution and adapting it to fit a given context.
Not saying leetcode is the answer, but it does solve for something things.
My interviews tend to walk through the framework of a project and they can speak to me in the abstract, or an example of what that part of the project does. Say there is a standard way of connecting to the database like in Rails. They can tell me about database.yml and how it has different entries for each of the databases. Then we have a conversation about checking in passwords to git, env variables, secrets managers, etc... This avoids asking stock questions which might be coached/studied more and aims for what the person doing the work might practically know. It also keeps the discussion in a context that makes understanding the questions (hopefully) easier. My style of interviews are very much non-standardized and there has to be some trust that I have some idea what I'm doing.
Leetcode at least has some standardization around the problem. Of course everyone could have looked them up and studied the exact solution, and these solutions don't correspond very well to daily efforts. But, given everyone knows the game, it does demonstrate some horsepower I guess. Or maybe I like these interviews because I'm good at them.
When I’m competing for one position and “we just want to see how you think” is always not true but instead a completely arbitrary set of criteria not presented to the candidate, it should be sanctionable
I HATE project interviews, because their skills aren't transferable to the next interview. Take home projects also tend to "go-over", because you want to put your best foot forward and you're betting that the other people they interview are also going to exceed time.
I also dislike those because I’ve already got a full time engineering job and a family and don’t have time to put 4-5 hours each into every 3-hour project the interviewers ask for. But 5 years ago I did.
I’ve concluded there really isn’t a single way to interview that checks all the boxes, and if you’re running the interview process you have to pay the costs somewhere, unfortunately.
IMHO, there is some risk of violating NDAs. For the most part companies don't care if you share how their tech stack works, but I get nervous about revealing IP.
ChatGTP is a growing problem but we have defenses against it (until its on device maybe?). Things like Frida are probably worse right now.
They dismissed my concerns and will continue as usual. A part of me wants cheating to be rampant to force companies hand in the matter. Unfortunately I'm with you and I suspect they will just enforce in-person, whiteboard interviews again (a colleague explicitly asked for it), rather than trying to come up with a better system.
There is nothing leetcode-specific about this cheating platform. OP just angled it that way because HN readers would lap it up that way.
Assuming someone was using this cheating platform, how would you run your interviews? Wouldn’t this screw up the actual legit interviews(whatever that is…) too?
I'm looking for people who can demonstrate that they have faced challenges and overcame them, that they communicate their decisions effectively, and that they can learn new information quickly, can receive and incorporate feedback, and so on.
Can't really cheat your way out of that.
I realize that this one project isn’t the only way to cheat in interviews, but I still think it’s naive to think that this tech will only harm what you perceive to be “bad interviews” and not affect your own preferred interviews. At the absolute minimum, it adds additional overhead to performing interviews where you have to also be aware and try to figure out if the interviewee is being coached like this.
Right now people "cheat" by using an IDE, but no one has trouble with that (and rightly so!).
So why should I care if someone is using LLMs to pass the interview and do their job if they are being successful?
A man can dream! I come across really badly on video. I hate the compressed audio, lack of body language, latency etc