Just as you said, I'm trying to make it a studio-ish product which combines various AI magics, so they can work together and create something that a single AI model can hardly achieve.
Here's my future plan: I'm going to integrate natural language generation, music generation, video generation... Into this project, so you can make something really cool with and only with AI!
And the most important part: it will always be completely open source and free, and (maybe) a Google Colab is enough to serve all these stuffs XD.
I have a hundred ideas of things I'd like to build with SD for cool drawing. There's a mile-long gap between doing things in Python and in GUIs. I'd like that to be an inch-long gap. For example:
- If I can use these tools in Python code (outside of your system, as a documented library), that's awesome
- If I can build plug-ins into your system, that's awesome
... and so on. Photoshop, and later gimp, sort-of did this with filters, and it was an amazing boost to both. What I'd really like is something like OpenProcessing meets carefree-creator, where I can easily do creative coding.
I have a neat model for how to use SD for rather creative, iterative artwork which I've wanted to hack, and I'd like a framework to hack it in.
That's a ton of great suggestions! I'll try to digest them and use them to guide my future works!
And for your requirement, actually carefree-creator is built to be a stand-alone backend server, so you can indeed customize your own frontend if you like, as I mentioned here (the first point): https://github.com/carefree0910/carefree-creator/issues/11#i...
I don't want a stand-along backend server with an AJAX API. I want a clean Python API on the back-end I can `pip install` (and, now that you mention it, a corresponding JavaScript on the front end). That should be wrapped in a thin layer to translate into a service-oriented API, which people might or might not use.
Performance issues aside (most of the integrations I'd want to build involve doing things between SD's iterative steps, and there would be a ton of data going back-and-forth):
- A service-oriented API only lets me use your code. I'd like to integrate with it to build new tools.
- Even if I want to use your code, a service-oriented API also doesn't let me do so practically. Adding your library to my code base involves launching a whole new server.
Flip it around: If you wanted to do a little bit of NLP in your tool, you'd probably be happy to `import spacy` or `import nltk`, add 5 commands, and use it. If you needed the devops of launch a server on a different tech stack, you'd probably make due without it.
And if you wanted to, in turn, extend `spacy` in some way, it's easy enough to toss in a few new things into it's pipeline, which, for example, annotate text. If you needed to launch a service to do so, it'd never be done.
Regarding documentation, my comments were architectural considerations. Your documentation is great for the stage you're at. I've basically never seen better documentation in a 6-week-old project since documentation make sense only once things have stabilized that they're not changing week-by-week. Otherwise, it's a lot of work for negative benefit. If people (like me) reuse or integrate with a 6-week-old project, you'll have pressure to support legacy APIs down-the-line. That only makes sense once those APIs stabilize. External contributions also make sense once the architecture has stabilized (which might or might not be the case 6-weeks-in).
This is an old project, with a VERY outdated document, but all the features of carefree-creator can be achieved with one or few lines of code from carefree-learn.
Take the typical text to image feature as an example, here's a piece of minimal-working-codes:
```python
import torch, cflearn
api = cflearn.cv.DiffusionAPI.from_sd("cuda:0", use_half=True)
api.txt2img("a beautiful, fantasy landscape", "out.png") # this will save the generated image to `out.png`
```
And you can install it with `pip install carefree-learn[cv]` XD.
Can I send you an email, or would that be excessive. I could describe what I'd like to build someday. Perhaps it might inform architecture, or perhaps not, depending on how it aligns with your vision.
Just as you said, I'm trying to make it a studio-ish product which combines various AI magics, so they can work together and create something that a single AI model can hardly achieve.
Here's my future plan: I'm going to integrate natural language generation, music generation, video generation... Into this project, so you can make something really cool with and only with AI!
And the most important part: it will always be completely open source and free, and (maybe) a Google Colab is enough to serve all these stuffs XD.