>We run a public buildbot that constantly runs new changes to our code against our test suite on every configuration...

In perforce terminology how is this done? You could use branches, but you could also shelve the change and have the build-bot unshelve it and test.

>We also have a commit queue, which is a way to try a change and have it landed automatically if the try succeeds

What happens if the change becomes incompatible due to other changes over the course of running the test? I assume the commit just fails in this case..