Using AI in your "traditional" artworks

[parkdalegardener]

Let's just toss this into img2img and tell the AI to do it's thing. I hit it pretty hard. See that Denoising strength slider. That's the "do your own thing" slider. The closer to 0 it is; the more the AI has to listen to what I say. In fact, at 0 the image would not change at all. If I set the slider to 1 I'm saying get as creative as you want. 0.95 is telling the AI I want a lot of creativity in it's interpretation of my lovely camels.

Well damn; this came out better on the first pass than I thought. Camels are facing the wrong way but the rest followed along fairly well. I can live with it. At this point if you wish you can add a depth map from a completely different composition such as a figure posed in a 3D program and a depth map saved from the project. Remember, it doesn't matter the sex, age, or clothing your figure was wearing; we've seen that we can change that at will.

[sanbie]

Wow, that's unbelievable pdg, just by doing that you got that image, wow, just wow 

[parkdalegardener]

I gave the AI almost carte blanche on that one. Usually I would work with much more control over the output by keeping the noise closer to zero and sending the results back over and over again for refinement till I got something closer to my original design. My lovely camels facing the other way.

img2img is just part of the whole pipeline to use in combination with inpainting and outpainting to achieve exactly the image you want. Those two things are up next.

[parkdalegardener]

Inpainting and outpainting are exactly like they sound. Painting inside an image, much like any paint program; or painting outside the borders of your original image. The first is pretty self explanatory. Take your image from what ever the source and paint adjustments into it via a transparency mask. This is the important part for both inpainting and outpainting. Your paint program must paint nothing where you want it to be transparent. That is one reason I use paint.net. Transparent is just that. Transparent. Some image editors/paint programs use black as a transparency colour. This won't work. Remember our depth masks from earlier. They are made of various amounts of opacity.

So here we go: Let's start with a young Pharaoh for our Egyptian themed contest entry. Here he is in glorious B&W. Now to illustrate a point we will give him striking baby blue eyes. He is a baby after all. We will send our child god image over to inpainting. It could just as easily be a Halloween photo I'm inpainting. Many of the online generation sites will let you inpaint and outpaint with images of your own. They do not need to be AI generated images in order to do this.

[parkdalegardener]

You can paint out the eyes in your paint program and upload the image. You could upload a mask with the eyes removed to place over the original image. I simply painted vague holes in the original image itself right in my web browser. Tell the AI you want bright blue eyes, or red bloodshot eyes, or green snake eyes. What ever you want. Hit the button and away you go. If you hit the button again it will give different bright blue eyes in the case of our mini Pharaoh.

[parkdalegardener]

Outpainting is the cool thing. There are various implementations available to try online but the basic idea is the same in all implementations. You get the AI to imagine what your image would look like if the canvas was extended in a given direction. We will take our boy king with the bright blue eyes and drop him in to inpainting/outpainting and ask the AI to extend the canvas upward and fill in the top of our pharaoh's headpiece.Et Viola, our young lad has had a top added to his headpiece. If you don't like it generate another. Changing the denoising will change the result. Outpainting has no size limitations in theory. It most certainly does in practice. The method you chose to do the outpainting and the machine doing the actual outpainting; either CPU or GPU will have RAM, VRAM and or other sizing limitations. If we sent our young contest entrant back thought the outpainting we could finish off each side of the headpiece and maybe bring down the bottom a few rows of pixels to get a more pleasing aspect ratio for our image.

Remember; you can go through img2img, depthmapping, in/outpainting and more over and over again with the help of your paint program refining your composition and details as you go. You can use them in any order you like or not use them at all. The great thing is that all of this stuff can work with your own images and designs. You can ask AI for an image and accept that as the output, but in reality it is more a tool to help with the creativity of the person using it. You will achieve way more artistic satisfaction and understanding of the limitations of the technology once you actually play around with it using AI as a creative tool and part of your workflow instead of letting AI image generation be the end result in and of itself.

[parkdalegardener]

Because I'm bored and have to wait around here for my lift to arrive; I'll sneak peek something I started working on. I've posted to the coffee shoppe a while back when I was first exploring this stuff. I was complaining about coherency and how hard it was to attain something passable. That was before depth mapping.

So the way this works is simple:

Use your video tool of choice to take your clip and break it into individual frames. I use the free ffmpeg. It doesn't matter what you use as long as you name the frames with sequential numbers and dump them all into a file folder somewhere.

Take a frame near the start of your video and upload it into the AI. Use the depth mapping model and describe your expected output. Hit generate and examine the output. If the description needs adjusting to get the desired outcome so be it. Adjust the prompt and try again. When you are happy with the image you are generating copy the seed as shown.

[parkdalegardener]

If we take that seed value and plug it into the AI at a later frame, somewhere near the middle of our video; we can check to be sure all is good or to adjust our prompt if necessary.

We would do that yet again for a frame near the end to quality check. If all looks good just run img2img in batch mode specifying your input images folder and where you want the output images to be placed.

It is then a simple matter of reassembling the frames with ffmpeg, or your editor of choice; back into a mp4, avi, or what have you.

I'll post a link to the output animation when it's done. My lift is here and I'm off.

[sanbie]

Very interesting read pdg

[parkdalegardener]

Well I won't show you the video generated. When I got back from dinner all was done. Sadly it is in the state of causing one visual harm with all the jittering. In a hurry to post before I went out yesterday I failed to follow my own advice and check a frame or two near the centre of the video. The little robin took a jump and spin while eating it's seed on the branch. This created a minor freak out for the AI. It did what it was supposed to do; but AI isn't intelligent despite the name and there is actual visual harm and torture to the poor little robin in the machine's output. We can have that. I'd have to start posting those disclaimers about no digital animals being harmed in the creation of the work.