> This has worked much better for us than asking a model to draw lines in a GUI. It also means the benchmark can spend less time testing computer use and more time testing electronics.

this is key I think - I use it for graphics tasks - it sucks at graphical environments - always use some text based representation, not really surprising I suppose. chatgpt even suggested I'll use the text representation, claude said how about I give you svg and you convert it, they seem to know.