Baseline testosterone levels are actually even lower in most hunter gatherer populations. This suggests that your explanation is not necessarily correct.