The other built a cascading prompt engineering pipeline that iteratively tries to improve on it's output in a series of nested steps. They're the ones hitting 5k runs per day because they've nested their automations so many layers deep.
This way of measuring things is kind of falling apart because people can nest automations and loop them to run many times on many values. One 'run' of a heavily nested automation could actually lead to hundreds of runs do to nesting.
My co-founder and I have debated this at length and the only way we see to solve this is to add a credit based system and stop caring about runs altogether. Each node would have a credit cost and each plan would have monthly credit alotment. If you can think of a better way we're very open to suggestions!
Can I ask how your runs are executed? I'm assuming tracking usage is probably trivial, if you're running each process in individual sandboxes.
Runs are basically dynamically generated scripts. Our backend parses the automation definition from the DAG on the frontend, fetches the definitions of each node and stitches it all together to run in a sandbox env. It started off quite simple in the early days but it's become quite an monstrous system with dynamic variables, nesting, looping, credential access and error handling.