When I set it up the goal was to avoid a lot of the issues I've personally had or heard about with coding challenges: they take way too long, they're too abstract, or you feel like the company might just be using you to get a couple of days of free work on their actual codebase. For this reason I feel fine about not paying people, as it's just a small part of the application process and not free work for us.
I think it's worked pretty well so far. It happens after the initial phone screen and before the first technical interview. We send people the challenge, they return it whenever they want, and if we like it we set up a technical interview. Since the challenge uses our tech stack and is similar to the work we actually do, a large part of the first technical interview is discussing their solution. Why they chose certain patterns, why they added a certain library, how they'd consider testing it, why a certain function might be slow with 10k entities, and so on.