yup, shame on me for that. thanks for headsup, this was addressed.
6 karma · joined July 28, 2020
We try to change software engineering hiring, so you never have to participate in the whiteboard interview ever again.
Feel free to reach out at ilya+hn at autoiterative dot com.
Thanks for the questions, let me elaborate:
- I didn't have time to prepare privacy policy, but we'll indeed make one and post it. You're right, we should have it.
- There is no tracking. Never will be. We'll blog about why we're for 100% anonymity of the candidates in more detail, but for now: We don't collect names. We don't require valid emails to enroll in the demo. We don't use tracking cookies. We don't send HTML emails with tracking pixels, and we don't use any 3rd party service. We use cookies to authenticate you, and that's it. Javascript, social media buttons, or trackers or analytics of any sort on our pages are considered a bug. What we don't have, is a proper explicit privacy policy explaining all that.
- We're a European company.
- That's how Namecheap sets it up -- curious, why do you think it is a bad thing?At this point, we decided that this is too good to hold only for ourselves. Thus, AutoIterative was born, where we also significantly improved the parts that weren't polished well enough and added features we lacked.
I would challenge the assumption that this is purely advertisement, though. In my head, this is not "buy our shit™" type of post -- in fact, there's no mention of buying the subscription anywhere. All there is is a link to a demo, with a valid challenge (not a fizzbuzz/fibonacci, but one you can actually have a lot of fun with), which we provide for free to anyone, w/o even requiring valid email -- it is up to you if you don't wanna reset the password later. Instead, the post is more of a "use the right metric -- here's our experience and what has worked for us" type of post.
Would you agree that this is a more fair assessment?
I fully agree with you about temp-to-hire or work-for-a-day models. And I think if the company has such a process in place, it is the sign of a well-developed culture. It is exactly the point of the blogpost -- evaluate using the right metric for the job. However, this is rare in practice, and I would say for the same reason the well-established culture is rare: it requires a lot of work to be done by the very same great people in advance. It is a chicken and egg problem. I saw countless times in multiple companies the signs of immaturity that prevent them from giving people access to their production: secrets in git, shared accounts, lack of established onboarding and offboarding processes, no processes for credential rotation, absence of audit logs, legal issues, you name it. The majority of companies can not afford a work-for-a-day option.
Maintaining the dedicated challenge is another option for them, but it has its downsides, too. In our experience, the main one is that you either invest in creating the grading infrastructure upfront or spend a lot of time manually grading the submissions. Checking the correctness of the code is a hard task unless one can actually run it and see how it performs under a barrage of tests. I'm not saying there should not be a code review, of course. I am saying that the automated assessment of the code should be done before you start spending human time. Machines are cheap, your engineers are not. And at this point, one doesn't need AutoIterative, but has effectively reinvented it :)
We walked this path ourselves over the years. We did fizzbuzz style interviews, that was suboptimal. We graded challenges manually for a long time, which was tedious and error-prone. We made all the mistakes and tried to fix them. We were using the wrong metric.
Qualitative improvement was when we started using the right metric, and automating its assessment was the next logical step.
And then it looks like this: when hiring someone, the company measures them by one dimension and then expects them to perform on another one. I did get some backlash for comparing engineers to cooks, heh, but the main point is and always was about choosing the right metric.
I would argue that 40-50 senior people in one place would be rather an exception from the norm, and your company would be quite an outlier on the bell curve. If all of you are not just seniors, but also had first-hand experience participating (or even building) in hiring loops of the large companies, then you would have all the necessary knowledge of why those rituals were created, and why. The point of the blog post was that this is usually not the case, so the imitation is the only option.
Could you elaborate on your hiring loop? Which of the stages is the most effective and brings you the most value?
Some context: we're not claiming we have a silver bullet or fit-it-all solution as some people got from the post (and that's my fault because I'm responsible for that text, and blogging is hard, haha). And we're also not claiming we can replace _the whole_ of your hiring loop (that would be the silver bullet).
We claim that choosing the right metric can drastically improve the hiring loop, and we've built the tool that automates the assessment of this exact metric.
I will try to address the questions tomorrow, I already see some of them require a ton of thinking indeed.
I want to share something we've built to improve the current state of hiring in our industry. We think algorithmic interviews are the wrong metric, so we're trying to have a better approach. We tested this idea successfully for more than a year in a ~2K people company, saw great results, and decided to make it into a product[1].
If I had to sum up the idea in one sentence, it would be this: "to hire great software engineers, test their ability to deliver something as close to real work as possible." Thus, we build a product that provides a challenge, a CI pipeline, and a "production environment" -- what the candidate needs to do is deliver something that works, making pipeline green and iterate. Sans peer review, this closely resembles the actual way it happens at their jobs. Behind the scenes, our system runs all sorts of tests against the submission, simulating the actual production. I can spend a lot of time talking about it, but we'll blog[2] about it in more detail in the following weeks.
We're currently in the super early stages, but we have a working demo that I want to share so that people can play with it, break it, criticize it and hopefully have as much fun with it as we had when building it. If you want to give us more direct feedback -- feel free to reach out over email to demo+hn at autoiterative.com.
[1]: Direct link to the demo: https://ais.autoiterative.com/demo [2]: Our blog: https://autoiterative.com/blog