No, it's not. Keep in mind in Linux, desktop environment means a big framework such as GNOME or KDE that runs on top of X or Wayland, i.e. a graphics server.

You can run bare X or Wayland, plus a window manager and cherry-picked daemons, to achieve the same sort of functionality e.g. desktop notifications, network roaming or device automounting.

My point is that bare X plus cherry-picked services tends to be much more efficient, because you don't need to pay a performance tax for the things you don't use.