It's a valid notion but is it supported by evidence? Supporting evidence would be evidence of periodic burning of villages because that's what warlords do to peaceful settlements.

Also it is likely that the notion of "elites" is different in our age than in the era before the invention of money. Our elites are not elites of mind, soul, or even body. They are sitting on their hordes of extracted wealth. Earlier any sort of elites in an agrarian society (which presumably precludes strongmen type of "elite") would likely be based on group psychology and charismatic power (shaman, priests, divine representative, etc.)