github.com
Hope that I can hear from you, any advice or suggestions will be really appreciated!
No! Your project is much cooler than the title suggests.
It’s like, some kind of AI art studio app that goes beyond just text prompts. It’s a natural seeming evolution towards using generative tech in more of an iterative / workflow way.
Great work.
Just as you said, I'm trying to make it a studio-ish product which combines various AI magics, so they can work together and create something that a single AI model can hardly achieve.
Here's my future plan: I'm going to integrate natural language generation, music generation, video generation... Into this project, so you can make something really cool with and only with AI!
And the most important part: it will always be completely open source and free, and (maybe) a Google Colab is enough to serve all these stuffs XD.
Integrate coding.
I have a hundred ideas of things I'd like to build with SD for cool drawing. There's a mile-long gap between doing things in Python and in GUIs. I'd like that to be an inch-long gap. For example:
- If I can use these tools in Python code (outside of your system, as a documented library), that's awesome
- If I can build plug-ins into your system, that's awesome
... and so on. Photoshop, and later gimp, sort-of did this with filters, and it was an amazing boost to both. What I'd really like is something like OpenProcessing meets carefree-creator, where I can easily do creative coding.
I have a neat model for how to use SD for rather creative, iterative artwork which I've wanted to hack, and I'd like a framework to hack it in.
And for your requirement, actually carefree-creator is built to be a stand-alone backend server, so you can indeed customize your own frontend if you like, as I mentioned here (the first point): https://github.com/carefree0910/carefree-creator/issues/11#i...
And if you are interested in customizing carefree-creator itself (like integrating your fancy model into it), I haven't written any documentation yet, but here are some brief guides: https://github.com/carefree0910/carefree-creator/issues/8#is...
Feel free to contact me if you encountered any trouble!
Performance issues aside (most of the integrations I'd want to build involve doing things between SD's iterative steps, and there would be a ton of data going back-and-forth):
- A service-oriented API only lets me use your code. I'd like to integrate with it to build new tools.
- Even if I want to use your code, a service-oriented API also doesn't let me do so practically. Adding your library to my code base involves launching a whole new server.
Flip it around: If you wanted to do a little bit of NLP in your tool, you'd probably be happy to `import spacy` or `import nltk`, add 5 commands, and use it. If you needed the devops of launch a server on a different tech stack, you'd probably make due without it.
And if you wanted to, in turn, extend `spacy` in some way, it's easy enough to toss in a few new things into it's pipeline, which, for example, annotate text. If you needed to launch a service to do so, it'd never be done.
Regarding documentation, my comments were architectural considerations. Your documentation is great for the stage you're at. I've basically never seen better documentation in a 6-week-old project since documentation make sense only once things have stabilized that they're not changing week-by-week. Otherwise, it's a lot of work for negative benefit. If people (like me) reuse or integrate with a 6-week-old project, you'll have pressure to support legacy APIs down-the-line. That only makes sense once those APIs stabilize. External contributions also make sense once the architecture has stabilized (which might or might not be the case 6-weeks-in).
If I didn't get you wrong, maybe another project (carefree-learn) meets your needs better: https://github.com/carefree0910/carefree-learn.
This is an old project, with a VERY outdated document, but all the features of carefree-creator can be achieved with one or few lines of code from carefree-learn.
Take the typical text to image feature as an example, here's a piece of minimal-working-codes:
```python
import torch, cflearn
api = cflearn.cv.DiffusionAPI.from_sd("cuda:0", use_half=True)
api.txt2img("a beautiful, fantasy landscape", "out.png") # this will save the generated image to `out.png`
```
And you can install it with `pip install carefree-learn[cv]` XD.
However, not for the purposes I described.
Can I send you an email, or would that be excessive. I could describe what I'd like to build someday. Perhaps it might inform architecture, or perhaps not, depending on how it aligns with your vision.
I've tried both the Docker and non-docker instructions. Using Docker I get an error stating "Could not find a version that satisfies the requirement onnx-simplifier>=0.4.1" (I'm running Python 3.9.6 per the installation instructions).
If I manually install the dependencies I got a bunch of errors whilst executing pip install.
Has anyone else experienced these errors and know how to go about resolving them? I'm on an m1 macbook.
Here's the onnx-simplifier project: https://pypi.org/project/onnx-simplifier/, looks like it already goes to 0.4.8.
I'll try to reproduce the error, before that maybe you can try the Google Colab solution:
https://colab.research.google.com/github/carefree0910/carefr...
I'm now getting different error messages by installing cmake (brew install cmake) before the manual install - it still looks like some odd error caused by Onnx, which is also why the manual install is failing - "fatal: not a git repository (or any of the parent directories): .git" whilst installing onnx.
The Docker solution failing is weird, containers are meant to be hermetic and deterministic, maybe an upstream change broke something..., but given the Docker install error messages and the fact Onnx is a large-scale project I fail to see how that's the case either.
I'll continue to debug and update the top comment if I resolve it :)
And since ONNX is not a forced requirement, maybe the following guides can make things easier:
1. Install `carefree-learn` instead of `carefree-learn[cv]`
2. Install the following packages manually:
ftfy regex lmdb albumentations carefree-cv
> I'll consider removing the `onnx` requirements from the `cv` spec in the near future!
The installation steps are as follows:
$ git clone https://github.com/carefree0910/carefree-creator
$ cd carefree-creator
$ pip3 install carefree-client carefree-learn carefree-cv ftfy regex lmdb albumentations kafka-python redis[hiredis] cos-python-sdk-v5 uvicorn torchvision
$ python3 -m uvicorn apis.interface:app --host 0.0.0.0 --port 8123
> *: The WebUI codes are not open sourced yet, but we are happy to open source them if it is truely helpful .
https://github.com/carefree0910/carefree-creator/issues/2
Apart from the technical concerns (huge code base, lack of documentation, etc.), I'm also pretty afraid of someone (or some company) just taking away the codes and make profits or even claim that they make it, because webui codes (especially as production ready as this project) are something that's easy to copy and HARD to protect...
But if more and more people get to know this project, this concern will no longer exist, and I'll definitely open source the webui codes XD!
After all, I do want more and more people to be able to enjoy the current AI magics, for free, and forever!
The MIT license is wrong for this, as is proprietary. You want AGPL.
They faced a similar concern when people were selling the exact same product using a new name.
> For example, this past year, we’ve had to deal with funded businesses plagiarizing or copying our work to directly compete with Sentry. This has included taking marketing content from our website, plagiarizing our documentation and framing it as their own, or straight-up copy/pasting our product visuals. Their defense? “Well, it’s free open-source, and we can do that.” These businesses are not using Sentry to improve how they develop software; they’re lifting its code and assets to build their closed-source products to compete directly with us.
But this really seems like something that would let me scratch my itch to produce nice illustrations, though ofc won't give the pleasure of learning that skill myself but surely stuff like this can't hurt. :3
Really cool project mate. :)
Currently the UX is not perfect, and I'll definitely try my best to improve it XD.
Looking forward to your advice and feedbacks!
I'll try to fix it ASAP!
(p.s. I personally recommend using this project on PC, because the features on the mobile site are very, very limited XD)
https://colab.research.google.com/github/carefree0910/carefr...
Which means you can use the Google Colab as the product's backend, so you can enjoy all the AI magics for free forever XD.
I'm also concerned that posting links of my creations will be some sort of spam...
Any advice will be really appreciated!
a lovely little cat, pixel art.
It will generate something weird, but (maybe) the quality of the generated image will be better with some tweaks!
However, I'm worried that a big part of these AI tools built around SD are being posted on GitHub, marketed as open source, accept contributions, but don't have any license.
This also means that technically speaking, you can't add the license after the fact without contacting (and getting the agreement of) all contributors that have ever contributed to your repo.
I'm also curious whether I should somehow follow the SD license... But I'm not actually using their codes though...
Any advice will be really appreciated!
Thanks for adding the license, very appreciated! This makes me want to take it for a spin much more!
Automatic on the other hand, seems actively opposed to responding to the calls (cries?) for him to clarify a license for the repo. Closed without being addressed - https://github.com/AUTOMATIC1111/stable-diffusion-webui/issu... 21 days open without response - https://github.com/AUTOMATIC1111/stable-diffusion-webui/issu...
We definitely need more projects who are completely open source (both WebUI and backend) to contribute to the ecosystem, and do so responsibly, adhering to lawful and ethical usage of the work of others.
Thanks a lot for the advice! It really helps!
</s>
The AI world does not seem to give a toss about licensing. I admire the spirit of "make stuff first, worry about copyright later", but you at least have to pay lip service to the system if you don't want to get roadkilled when "later" comes too quick.
If you're on GitHub there is a fallback license in the TOS that covers forking and PRs, but you have nothing to protect you if you go off-site for whatever reason. It's there mainly to keep their users from getting sued for just being on GitHub. I don't remember how it interacts with multiple collaborating users all under the same fallback license; you may be able to argue implied license in that case, but implied license is even narrower than the GitHub fallback license.
I think you've mentioned something important: I haven't got enough time practicing. Maybe when I have more free time, I'll try to pick up these skills XD!
There is no such thing as "better" drawing. Drawing cannot be categorised as "better" or "worse". Just have a look at paintings of some of the well-known artists, example Picasso, and you will know what I mean.
By saying 'better', I'm not actually 'willing' to become better. In fact, I enjoy drawing even if the outcomes are 'bad'.
The hard part is, when I want to draw something 'concrete', I can't... For example, a story between a boy and a girl - The only thing I can do is to draw two sticks in order to make them consistent frame by frame XD.
So yes, I have to admit that I still want to share my creations with others, and without being better I can hardly do so... I think artists like Picasso are at another level, because they might treat art as their whole life, and might have the ability to focus and only focus on their own arts.
And me, as a normal person, will still need some compliment XD.
If you ever take an intro art class, you will spend a lot of time doing boring exercises like drawing a bowl of fruit, again and again.
This teaches you how to really "see" something, how to understand the shape, lighting, the true details. When we look at a thing we typically briefly glance at it long enough for our brain to categorize it, and then we move on. To get good at drawing, you need to train your brain to truly see things.
Then you learn how to hold something in your mind's eye and draw it. You start by briefly glancing away, drawing, then you look away for longer and longer, and keep drawing.
Drawing is not a mechanical skill of your hands, it is a mental skill.
Picasso was a very good artist, look at some of his “better“ works where he was just doing a portrait or something and he’s up there with any other artist known to history.
You kind of have to be a really good technical artist to be a really good “bad” artist.