I will try to address the questions tomorrow, I already see some of them require a ton of thinking indeed.
At this point, we decided that this is too good to hold only for ourselves. Thus, AutoIterative was born, where we also significantly improved the parts that weren't polished well enough and added features we lacked.
Some context: we're not claiming we have a silver bullet or fit-it-all solution as some people got from the post (and that's my fault because I'm responsible for that text, and blogging is hard, haha). And we're also not claiming we can replace _the whole_ of your hiring loop (that would be the silver bullet).
We claim that choosing the right metric can drastically improve the hiring loop, and we've built the tool that automates the assessment of this exact metric.