It very much is a skill. You wont get good results by asking it to "go do X" and walking away from the desk. You have to know how to use it as a tool to: 1) research the task 2) develop a plan the agent can follow to complete the task 3) check the plan for completeness/issues

After a (potentially long) session of investigation and planning _then_ you let it code. Even just being aware of this process is part of the skill involved (then you have to actually get used to doing it effectively).