More exploration of Stable Diffusion using a graphic novel style. This required further training of the AI and the use of a fine tuned dreambooth model. I actually really like this style and can see someone using it in a music video, which is what I'm aiming for as the goal for this study.
There's a lot of small inter related variables in play with this AI stuff. Starting with the output resolution which I have noticed can drastically change the image (Just because you have something that looks one way at 1025x576, it could look quite different at 1280x720). This along with the strength of the applied effect and the samples used (and sampling method) can all influence the output to a significant degree.
It's a lot of fun experimenting and totally wild when you hit a sweet spot but there's also a lot of trial and error with frequent odd results and iteration.
Also, I'm running all of this online using Google Colab so I am at the whim of available resources because I'm not actually paying anything to use it. If I were using this for a client, I would need to invest in the service or upgrading a computer here and take it all offline.
It also takes time to render each frame. For my experiments I am assuming a 12FPS playback so only analysing and producing every other frame. Also, for this example, I'm working at 1024x576, then scaling up in after Effects. This is to keep the render time from Stable Diffusion down. I guess it's about 15-20 seconds per frame to analyse and produce which is not bad at all as far as I'm concerned, I often wait 2 or 3 minutes per frame to render something in 3d using Redshift (that's about the limit of my patience, if it takes any longer, I generally find another way of doing something).
I think this will be a challenge to art direct.
Comments