Hello everyone, hope you are doing fine. I haven’t been able to post much recently as I was training and testing 2 custom AI diffusion models this last week and a half. For more info on how you can create your own AI diffusion model, take a look at this excellent tutorial by @kaliyuga.
Here are some images from my first AI diffusion model:
These are all upscaled using Swinir.
KIDLIT Diffusion
Here are some images from my 2nd model which we (my wife and I) call kidlit diffusion model. The dataset which I used for training this model was entirely from my wife's digital art which she made using Procreate. Here are some generated images:
I am absolutely in love with the images produced with the KIDLITDiffusion model!
Closing Thoughts
When are custom diffusion models needed?
I feel when you have very specific requirements, it is better to train your own model with the appropriate dataset. Kaliyuga's pixel art diffusion notebook is an excellent example.
On the other hand, if you are just starting out or your needs are more varied, better stick to the open ai diffusion models and explore first.
Some tips while training
I am certainly not an expert but these are things I noticed while training my models.
Train with the highest quality images you have and which you can afford to train on - higher quality images produce larger size training checkpoints and I think they take longer too. The upside is you can probably get away with lesser training steps.
As you can see in the images produced with my first custom model, they are not of great quality, especially after upscaling. This is because I trained the model with just 256x256 sized images. Also, if i try to generate an image with a higher resolution than that, I run into tiling issues.
On the other hand, with my second model, the KIDLIT Diffusion one, I can tell you that most of the images have not been upscaled. I rendered at 1024 x 768 too and there were no tiling issues.
I think this is because I did not resize images this time while training. Many of the images were 2500x2500 and some even higher. I left them as they were and you can see the quality of the output. What you sow is what you reap is very true here.
Also, it is better to crop your dataset images into square formats.
These are my conclusions from my experiences. I might be very wrong about some things; as I said I am not an expert. If someone notices something wrong here, please let me know in the comments.
That's all for today. Hope you found this useful and entertaining. See you in the next one!
Do you want to generate similar art with the help of AI (or maybe something entirely different)? It is quite simple to get started. You can get started here.
linktr.ee
My NFTs on NFT Showroom
My Splinterlands Referral Link