The general formula is (r^x) * (1/x) + (1 - r^x) * (1 + 1/x), where r is the percentage of negative tests and x is the number to mix.
With an 85% negative rate, it's more beneficial to mix 3 tests at a time for 0.72 tests per sample. (2 samples gives 0.78 tests, 4 gives 0.73) As more samples come back negative it becomes more advantageous to mix more, but you shouldn't mix 10 tests at a time (as opposed to 9) till you get to about a 96% negative rate, at which point you're running 0.44 tests per sample.
Edit: The binary search algorithm mentioned elsewhere would probably be more optimal, but I'm gonna do my day job instead of figuring out the dynamics of that one.
If you include that your goal is to test everyone too, I have no doubt that you can easily reach 96% negative test quite easily.
I remember in Quebec while we were only testing people with symptoms and that had traveled (or have been in contact with someone that has been tested positive), we were still way higher than 90% of negative tests (even right now we are a 90.1%