This doesn't work for anything that doesn't use the upstream default stable diffusion checkpoint. I generate a lot of images with Pastel-mix, Anything, Waifu Diffusion and Counterfeit, and none of those are giving sensible results with this tool.
The model underneath this is trained only on data from SD 1.4/5 so this would be expected. I have another model training which covers all models you mentioned which should perform well on those
I'd be interested to see the results of that. My email address is on my Hacker News profile if you want to talk there.
Can I have some kind of direct contact with you? Email, Twitter DM, Telegram, heck, I would download some new kind of app just to have some conversations.