Hi guys! As some of you know, I've been involved in creating AI art for almost two years. When I'm not building my own models, I use a lot of different Google Colab notebooks in my art. Even though there are a lot of really excellent notebooks available, tracking them down can be a challenge. I haven't seen a really comprehensive list of the ones used by the nascent AI art community, so I figured I'd put one together and give more people the freedom to create. This post will continue to grow as I find new notebooks, and as new methods are created. If you find any I've missed or have created any, let me know and I'll add them!
Table of Contents
- The Big Sleep
- VQGAN + Clip
2a. VQGAN+CLIP with Video Features - Clip-Guided Diffusion
- Styledream Notebook
- Pixray PixelDraw
- Real-ESRGAN Practical Image Restoration
- ruDALLE [Added Nov 2, 2021; Updated Nov 3, 2021]
- Looking Glass AI [Added Dec. 28 2021]
- JAX Clip-Guided Diffusion *[Added Mar. 11, 2022]
- CLIP Guided Deep Image Prior *[Added Mar. 11, 2022]
- Disco Diffusion [Added Mar. 11, 2022]
- Disco Diffusion Turbo [Added Mar. 11, 2022]
1. The Big Sleep
A Beautiful House by BigSleep
Description: The original Clip-guided AI Colab tool; uses Clip to guide BigGAN. Fairly outdated, but absolutely foundational to the entire art field.
Author: Ryan Murdock (advadnoun on Twitter)
Notebook Link
2. VQGAN + Clip
Gordian Gallery by KaliYuga, 2021
Description:
By far the most widely-used of the Clip-guided methods[Edit April 2022: very few people still seem to use this tool, but it's a foundational and very important one]; uses Clip to guide VQGAN.
Author: Katherine Crowson (RiversHaveWings on twitter)
Notebook Link (The notebook is in Spanish, but is easily translated to English using Chrome).
2a. VQGAN+CLIP with Video Features
Description: What it says on the tin! VQGAN + Clip with added video features and also some really good demystifications of parameters.
Author: @jotakrevs
Notebook Link
3. Clip-Guided Diffusion
Psychic Sermon in the Temple of the Mind by KaliYuga, 2021
Description: Produces absolutely unbelievable results, but very slow and resource-heavy. Uses Clip to guide a Diffusion model.
Author: Katherine Crowson (RiversHaveWings on twitter)
Notebook Link
4. Styledream Notebook
Portraits of DC heroes in the style of Alex Ross *by .hack//HAMBRE
Description: Portraits only! Clip-guided StyleGAN model.
Author: Ariel Ekgren (ArYoMo on twitter)
Notebook Link
5. Pixray PixelDraw
A synthwave great white shark by Viktor Alm
Description: A fork of NerdyRodent's fork of RiversHaveWings'VQGAN + Clip Notebook; uses VQGAN + Clip to produce pretty impressive pixel art.
Author:Dribnet
Notebook Link
Extra! You can also start with an initial image in a different version of Pixray; if you use an image originally created by pixray, the results can be pretty trippy! For example:
Via the Colab Notebook's demo params
Other bits and pieces of this project can be found here.
6. Real-ESRGAN Practical Image Restoration
Examples via the Colab Notebook
Description: A demo of Real-ESRGAN,which repairs many types of image degradation and also upscales the image 4x.
Authors: Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan
Notebook Link
7. ruDALLE
Description: A Fully-Trained, Accessible, Multi-Billion-Parameter Answer to OpenAI's DALL-E Model (in Russian)
Authors: Sber AI and Sber Devices
Notebook Link
Bonus: A faster Colab notebook for free users
8. Looking Glass AI
"Untitled" by KaliYuga, 2021
Description: An Accessible One-Shot Finetuner For RuDalle
Authors: AI_Curio and Others
Notebook Link
9. JAX Clip-Guided Diffusion
By KaliYuga, February 2022
Description: [Via Gituhub] "The models are denoising diffusion probabilistic models (https://arxiv.org/abs/2006.11239), which are trained to reverse a gradual noising process, allowing the models to generate samples from the learned data distributions starting from random noise."
Authors: Katherine Crowson and John David Pressman
Notebook Link
10. CLIP Guided Deep Image Prior
By KaliYuga, February 2022
Description: Creates soft, print-like images quite unlike what you generally get out of other CLIP-guided diffusion models
Authors: Daniel Russell, based on work by Katherine Crowson and Ryan Murdock
Notebook Link
11. Disco Diffusion
By KaliYuga, March 2022
Description: My daily drive; Very high quality image results very quickly; comes fully loaded with 3D animation capabilities and the latest Clip models.
Authors: Somnai and Adam Letts
Notebook Link
12. Disco Diffusion Turbo
https://twitter.com/KaliYuga_ai/status/1502057904693723137?s=20&t=MLWq2-K9-Rc7Bu5IN9wZWg
- KaliYuga_ai
Description: A fork of Disco which speeds up and cleans up 3D animation significantly. I also use this all the time!
Author: Chris Allen
Notebook Link
Bonus by Chris Allen Notebook: Wiggle Standalone Keyframe Generator; lets you generate keyframe instructions that give you fine control over Turbo's motion.