we're developing https://copperbench.org - an open-source benchmark for evaluating ai agents on real, verifiable hardware design tasks.

I do not have experience with Copperhead, but I made 2 PCBs with Claude. He made a py script that generates Kicad sources, I ended up generating all files (in particular schematics and PCB) from a shared netlist, and use Kicad cli to validate and generate gerbers.

I manufactured both PCBs via PCBWay (Claude also red all the specs and configured the constraints accordingly) and both work. I'm pretty impressed, especially because I've never done a PCB myself and didn't know I could.

What I was doing manually, co-working with Claude, was placing and routing (I was doing those via Kicad, then saving, then Claude was re-importing my work in his py files).

I recommend having the LLM write Skidl, which generates a netlist, which you import in the PCB editor, which the LLM then reviews. Routing itself I think would require code-driven routing (expressing board via layout constraints, kinda like UI constraints/CSS) which does not exist yet.

Copperhead directly edits the KiCad files so that is the CLI version of that Astra demo.

This is what I want to know as well. I'm about to kick off a project or two.