I didn't expect there to be a time limit and I do know benchmarks that don't have a time limit. They just penalize slower models which I think is fair.

In my experience models just don't take forever to mark tasks as done.

For my coding usage I don't set time limit so benchmarks that do so are providing me less interesting use cases.

With that said, I can understand having budget limits for expensive LLMs for those of us that don't have infinite VC money.

As for no internet access, the issue is that for most problems we do want Llms to be able to search docs, API specs, GitHub issues, etc. So not allowing that usually just favours larger LLMs that were able to memorize more data, not necessarily smarter ones when both have internet access.