Hacker News Reader

Comment by mk_stjames | Hacker News Reader

I don't see how the loading works for the end user's custom dataset. In fact, I find the layers of abstraction you have between getting the finetuning dataset and the actual training very opaque. I can't even tell where the dataset is coming from, it doesn't appear to be an example local to this repository.

I think a lot of people what something like... "drop .txt files of example data to train on in this /folder/ and run python finetune.py /folder/

View on HN

You are right in that the loading is right now on huggingface’s dataset. The feedback about it being opaque has merit, and we are working on giving users more control and visibility into the dataset loading. To your point, adding instructions about how one can load their own dataset and do fine-tuning can assist researchers better in leveraging these models. That being said, the README under each model folder has all the info one needs to get started.

More than happy to have you contribute to the repo. There’s a lot of exciting work to be done.

It's hardcoded, hidden within the code there are calls to huggingface's datasets.load_dataset, but you don't get to specify on your own..

Great observation. We are working on making this part very explicit. The goal was to let researchers get up to speed with the codebase to begin with, and then they would understand what needs to change to make these models work on custom datasets.

That being said, we are working on adding instructions to specify dataset and also the prompt that users want to use.

This is actually what I was hoping for. For Web UI that you can load a model then load some data and hit train.

You can do this in the stable defusion UI to fine tune models with your own dataset

OobaBooga supports this kind of load-and-go LORA: https://github.com/oobabooga/text-generation-webui

Looked at the project. Great initiative.

Feedback taken. We are working on making it more explicit for users to mention the task and dataset they want to train models on. Additionally, we will introduce a flag to let people mention the prompt they want to use for finetuning these models.

Reply on news.ycombinator.com