This is an ad. You'd be best served avoiding additional dependencies. At this point, you don't want to be trading off simplicity for ease. Even transformers + huggingface feels like too much bloat.

You can use this https://github.com/PygmalionAI/training-code

Or, you can use this; for QLoRA https://github.com/artidoro/qlora

The tools and mechanisms to get a model to do what you want is ever so changing, ever so quickly. Build and understand a notebook yourself, and reduce dependencies. You will need to switch them.