Why wouldn't an LLM be able to do high speed designs? Are you inventing new types of design rules for every board? Or is it simply because the automatic layout tools are not up to the task yet?
There a lots of things that can be subtly wrong, and it gets much worse as your signal frequency increases, or you have to deal with high power/radio/etc. The model (agent) has no way to validate its work.