The participants in the study were IT professionals that managed servers for a living. Servers and actions regarding servers were definitely within their domain experience. So we did exactly what you were suggesting we did.

I still think that connecting up a hundred little boxes is just as complicated as actually looking at code syntax, but even more so since everything is a black box. How do you suggest people fix errors in the output when the only action they can take is to connect boxes with lines?