> The Roman/Byzantine empire was pretty much destroyed by Muslims, though

This is hilariously untrue, if one is talking about the Western Roman empire. An islamic state did conquer Constantinople, which I presume is what you mean.

The Roman empire (western) basically fell apart due to infighting more than anything else.