Why isn't there a framework that does all this automatically for you?

I tried torch FSDP but it only managed to increase the memory to something like 150% of 1 GPU.

I eventually ended up sharding my model manually with .cuda() and .to() which works much better, but now I am limited to one module on one GPU and I would like to expand even more, and that would mean spinning up more nodes and splitting the model over how many GPUs manually.

I would be interested if anyone knows of a framework that manages this automatically and just works.

EDIT: BTW I am talking about model sharding not data parallelism which works very well with DDP.