And this is just me guessing from the article, there are other reasons to think twice before reaching such conclusions. The concept of "slave" is not as set in stone as we think, it's a generalization of a broader concept.
And cattle is not necessarily for the rich: livestock is a pretty good way to store energy. In hot places like the tropics, vegetables go ripe very quickly so the logistics of feeding an army of builders with crops might have been difficult enough that it was just easier to feed the crops to cattle then move the cattle to the people.