That's not a good comparison because it would be even harder to write your own X server. And yet X is still extremely fragmented.

And developing a new compositor also does not require wlroots. Originally there was libweston, wlroots is just another option.