It's always interesting to see you workflow process. I'm doing something similar today, in terms of pay with init images. What I'm doing is use dall-e for quick random outputs from a given prompt, to get the "sketches" for the artwork series. Then I upscale them in ESRGAN, it sharpens them and make pop out the detail. Now they are baking in the SME notebook, with the same prompt used in dall-e.
Regards and take this !PIZZA!
Cool. Eager to see your result. Besides the 256x256 restriction, i find Dall-e quite inventive for textures, shapes, and quite coherent with the prompt. but still it lacks detail but that's due to size. What's the ratio you upscale with ERSRGAN, i am thinking that maybe i am upscaling too much and at wrong time ? Did'nt you encounter a Memory issue by feeding SME notebook with a 4Mb upscaled picture ? Well i need to try. Usually upscaling it my last steps... Never tought to upscale during the process. Good idea.
Well, I upscaled them at 3.5x, so got resolution of 812 pixels aprox. I'm currently using a P100 but when I was using the K80, upscale from 500pixels to 2K was not a problem.
Another matter is what you give to the VQGAN or MSE notebooks. For a K80 more than 500x500 is too much. For a T4 more than 600x600 is a crash, and P100 can bear 700x700.
What I would do is just upscale them, and then reduce the size in a edition program to match the max resolution that the GPU given can bear.