In some environments, not-building stuff you might want to do could include launching/supervising long-lived processes (services), connecting networked filesystems and mounting them somewhere appropriate, firing up VMs, etc. Some of this you can definitely do with direnv but if you want reproducibility with those aspects of your environment, building it on top of Nix seems like a more natural fit to me.

There are other tools that compete in this space right now (devenv.sh, flake.parts, devbox), and for simple use cases a basic direnv integration might be just fine. But the role Flox is aiming for here seems pretty clear imo.