You can move the burden of disposing to the caller (return the disposable object and let the caller put it in a using statement).
In addition, if the caller itself is a long-lived object it can remember the object and implement dispose itself by delegating. Then the user of the long-lived object can manage it.