The reason I an ask is, it would felt like a 5 minutes task, but I track my time and found out often time I thought I’d just quickly check the progress made by the agents and it would easily becomes a 10, 15, or even a 30 minutes task.
Nobody could get more than 3 of them. Most people were shocked that 5 minutes was up already. My coworker who did interviews for our company was shaken that he had been judging applicants too harshly after he couldn't finish.
They were trivial problems. But 5 minutes is a very short amount of time.
I experienced both kind of sessions, one kind is like very complicated thing and it finish thousands of lines on its own for quite a while (long horizon problem), another kind is like a seemingly simple task that the agent done in a minute but then I need a few back and forth to get it right easily taking me 30 minutes of my time.
The former kind of experience can make us misjudge how much time we think a task would take us (with agents) to do. And then when the second kind happens, it would be quite disrupting as now we felt like it is delaying our progress.
So tracking the time taken when the second kind happens can help us calibrating what we can expect. I mean if we’re lucky it might take us no time but then we can’t expect being lucky all the time.