sahil_chaudhary

Dropping the batch size

Yeah the license issue is the only reason why I didn't release the weights yet.

I'm training a 65B model right now, also I believe you can use lora-alpaca to train on this data on a much smaller machine.

Given how many people are asking for the weights, I'm working to make them available now. So you'll be able to run them locally as well.

Yeah the weights did change.

All included it costs under 70$ for the 13B model. Training 65B now so will report what that will cost.

Interesting, in my experiments alpaca-lora performed worse but definitely possible that isn't the case generally.

That was an issue on my end, should be fine now

You can find my email on my HN profile

Ah yes, this model treats this purely as image captioning. The model isn't open source yet.

Nice, does this use clip-interrogator?

Glad you enjoy it! It hasn't been trained on non-generated images so is unpredictable when uploading a real photo, specially one with people in it

The model was trained exclusively on stable diffusion generated images, so can be unpredictable with non-generated images, specially images with people in it

The dataset used to train this model didn’t have any img2img data so that would explain it

Agreed, results not good for this style of images. I have a model training on a much bigger dataset of image-prompt pairs which should perform better on this.

The model underneath this is trained only on data from SD 1.4/5 so this would be expected. I have another model training which covers all models you mentioned which should perform well on those

Interesting, it hasn’t been tested extensively for non-generated images.

Not yet, looking into creating a write up on how it works and possibly open-sourcing

Shoutout to https://banana.dev , couldn't have made this demo without their hosting

It is actually based on a different approach, it uses an image-captioning model finetuned on image-prompt pairs

Glad you like it

Nice, that's the aim, not to get the actual prompt back but get a prompt which can generate same style of images

Glad you enjoy it

True, images generated through some UIs have prompts in meta data, aim here is to work on images people find online with no metadata. So it doesn't try to read the metadata but actually predict a similar prompt

It is based on an image-captioning model, so a different approach then CLIP interrogator, though you are correct that aim is not to get the exact prompt back but actually get a prompt to generate similar styles of images

It works by using an image-captioning model finetuned on SD prompts, so it may be outputting common known artists based on their occurrence in the training data

It actually works on top of an image captioning model, SD takes in keywords as well like "artstation" and "octane render" which are not covered in standard captioning so that is why the difference between using an off-the-shelf captioning model vs this

tested with png and jpg, doesn't work with webp yet

Hey HN! Img2Prompt lets you generate prompts from stable diffusion generated images, to help you prompt engineer specific styles and images.

Would love to hear your thoughts and happy to answer questions.

Hey all! Quick colab to try out Img2Prompt, get accurate prompts from stable diffusion generated images. Works with stable diffusion v1.4/5 images for now