It used to use more resources and was slower than Gnome 2, when it was still a thing. I don’t know why it got a reputation of being lightweight. I guess people assume less features mean better optimized, which is jot really true. I used to run it for years though, really liked it for a while.
I think it's because Xubuntu was a lighter alternative to Ubuntu when it ran Unity desktop. I don't know how Gnome compares to Unity, but at the time Xubuntu was far better for low-end computers than vanilla Ubuntu so the reputation of XFCE was justified.
Xubuntu was lighter than Ubuntu (when it had Gnome), at least that is how I remember it.