Fantasies Attic
Programs and more Programs => The "AI" Generation => Topic started by: parkdalegardener on January 03, 2023, 07:08:18 PM
-
So; we have an AI gallery now. I think people don't truly grasp the depth of what AI can mean to a non AI artist. I see lots of misinformation and fear about the whole scene. I'm not going to lecture one way or the other. I do think that if people saw there was a way to incorporate it into their work flow there would be better understanding of the whole thing rather than an outright opinion. Reddit and Twitter are quite toxic over this and unfounded opinions are flying fast and furious masquerading as fact.
So let's put this AI thing to some practical use. Egypt is a current contest theme. Poser, Blender, and probably DS; all have a cylinder prop of some type. Now I don't happen to have an Egyptian Column or a texture for one but I have that cylinder prop. I want an Egyptian column or two for use in an entry. Let's do AI to make a few.
So; I run Stable Diffusion locally. I'm cheap and can't afford online generations so I simply fire up SD in another browser tab. I'm doing this in real time. Typing as I go. Simple prompt "Egyptian hieroglyphics, orthographic" and set the output to seamless tile. In another tab 12 test images are generating... and they are done. Pardon me for a sec while I take a peek. Good. There are a few usable ones.
OK, I like one of these. so I'll send it to another browser tab. An open source AI tool for generating normal maps. Done. A normal map. Instant. AI strikes again. Download your normal map and there you go.
There you go. An image map with a normal that you can use to texture your new Egyptian Column.
The whole process was done faster than I could type this out. That is the real thing about AI generated processes. Sheer speed.
Oh; and the paint program I use is another open source freebie paint.net
-
I have always loved your tutorials pdg, I know you taught me a lot...most of it I have now forgotten and needs a memory jog to click back into place lol
I had to stop using poser as the print was far too small and my eyes got so sore real quick.
-
Then I'll just have to use bigger pictures sanbie.
-
You can make most any seamless texture. Brick walls, pillow embroidery, dress material. All of it without burnt in lighting effects. Scale as necessary to suit the size of the object you want to texture. Generation information is in the image name by default.
-
Even came up with a pale "freen" background I see. :(ROFLMAO:
-
Love the freen pdg lol
-
I see a lot of people making backing plates for their foreground figures. Simple enough but you can also use your images for environments. Accurate reflections and all that jazz. No problem. Equirectangular and most other environment spheres are mapped 2:1. If you asymmetrically tile your output to that scale you will get an image suitable for mapping to your environment. 360 degree reflections on your windshield glass, sci-fi visor, metal armour and so on. If you need the image to light the scene, Blender amongs't other free and open source software can easily provide light probes. Poser can use the image to light a scene as is but it could use a little work in paint.net to crank out reasonable lighting. Still, any environment you want, for any imagined scene; from ancient history to a dystopian future to deep space is there if you can imagine it.
-
I've got a feeling this room will get a lot of visitors PDG....Thanks for doing this...
-
Any time.
-
I didn't know about the second AI you talked about for seamless files. Since the time I tried to convince Stable Diffusion (online and with only 4 choices at time) to do seamless work!
:ty:
Unfortunately, the egyptian motifs weren't as good as yours when I tried :tearlaugh: It was more a kind of "went in washing machine" :tearlaugh:
-
I haven't "cherry picked the results" as they say. Straight up output from the prompt just to show the workflow and possibilities. If I want to really refine a request (prompt) I run a grid similar to the "contact sheets" and "exposure strips" from the old days of analog print film and darkroom work.
It's not the same thing as the mass generation of images I put out for the Egyptian textures. That is the point and spray technique. Roll the dice and take your chances. A very crude but effective method of generating a bunch of textures all at once.
-
So; we've seen we can use AI for making textures and the normals for them.
We can make any environment, though getting perfect environmental lighting is more specific to the software you want to use the environment in.
We can make any background plate we need.
Not really anything you couldn't do in a good paint program. That's the point. AI is simply a tool. Just like Poser or photography or paint.net.
So what else can we do with it? Most of us are familiar with depth maps or z-depth mapping or f-stops in photography. 3D artists have an option to deal with them in their software. Usually they are used to blur out a background or soften near foreground objects. They can even be used to mask out unwanted parts of an image.
2D artists have had no access to such, and after the creation of an image; neither do 3D artists.
Enter MiDaS. MiDaS is an AI model that estimates monocular depth. Huh? It guesses how close or far away something is from a camera. It was developed for autonomous driving. We will use it a little differently. We will use it to estimate the depth in a 2D image.
For instance, I can change out a character at the last second without changing my composition. I generated a portrait of a cougar bard for a game character. At the last second the character wanted to be a fox warrior instead. I'll show you the generation of the bard and then the change to warrior.
The bard image does not have to come from Stable Diffusion. I just show the generation for clarity on the whole thing. In the upper left of the screen you see I used the 2.1 768 model to generate the original image. I then sent my cat picture to image to image tab.
In the second screen grab you can see I have changed to the MiDaS depth mapping model. It has a smaller base size of 512x512 in comparison to the larger 768x768 model used to generate our cat picture.
What I am attempting to illustrate is that the origin of the picture you start with doesn't matter. Just upload it into img2img. Make sure you are using the MiDaS depth mapping. Tell the AI what you really want is a warrior fox. The AI will give you a warrior fox in the same pose (mostly) keeping as much coherence as possible. It works the other way as well to replace the background should you wish.
-
Couldn't add the second image for some reason
-
Boy that just went right over my head lmao
Will have to read it at a later date in the hope it goes into this old brain lol
-
For folk that use a 3D program to create art; there is a depth map render pass that we are all familiar with, though not all actually have a use for it. For people that have 2D art like a photograph or a render of 3D artwork and have no such map for a particular scene; MiDaS can generate one.
Stable Diffusion, dall-e, Midjourney, Wombo, or what have you all have two parts. The first part works by turning words into tokens that represent those words; and the second part then combines the tokens hopefully into something resembling what the user requested. Results are not guaranteed by any stretch of the imagination. That's why it's hard to generate similar images across online services. Each wishes to massage the data behind the scenes. The results from Disco Diffusion look different than Midjourney and so on even though they do the same thing using the same basic model to start with.
Some of the online generation services allow you to choose from the original model files (stable diffusion by Stability AI and partners) and others. I used a MiDaS model to illustrate how powerful depth mapping with this model is. I changed the entire character from cat to dog and outfits from one type to another as easily as shown. Yes image editors and paint programs can do it; but with great effort and skill. You can use auto-masking in Photoshop and other programs. Exactly the same type of thing but you are not training Adobe to get smarter simply by using their ai software.
In fact Adobe now sells an AI plugin that is stable diffusion model. There are also free plugins to use stable diffusion in Blender or the open source Krita. All three can use the MiDaS model. It's kind of auto-magical. I didn't have to generate a mask for anything. Clothing change for instance. I could have left the animal unchanged but the clothing recoloured.
All this is getting more confusing no doubt but it does lead somewhere. I'll have to explain depth mapping with different images. I'm going to get my act together here and be back in a bit.
-
Let me try this.
I want to make a picture in my 3D software for the current contest. A couple of statues of the god Horus guarding a path to some pyramids in the distance. One of the options when I render my image is to render a depth map as well. This is true of Blender, Poser, and I am pretty sure; DS. If I rendered my picture and the depth map and put them side by side they would look like this.
-
Pretty straight forward. Your image and a grey scale copy that shows how close to, or far away from; the viewer a thing is. The big statue is close to us in the foreground. Another big statue is in the midground, and some pyramids are in the background.
If we take our depth map and darken up the contrast in paint.net or whatever your paint program/image editor of choice may be; and paint a little black where we want everything to be left alone, we can mask out our two statues from the background.
-
Yep, I understand that...
-
As you may have noticed by now; stable diffusion has absolutely no idea of Egyptian deities. Horus, Ra, Seth, and so on are all just pharaohs. The mask still needs a bit of work before it's ready for prime time but I can paint a hawk god in place of the statues supplied in my paint program.
If I was starting with a photograph I took on a backpack through Africa trip when I was a teenager, and wanted to use it to supply the background for a separate composition for a contest entry; then the masking to get rid of the foreground would be even more time consuming as you have no depth map to start with.
The MiDaS model was trained for autonomous driving after all so that's the secret. It inputs a 2D colour image from a camera, and as quickly as possible; separates everything it sees from it's surroundings, and then determines how close each object is to becoming a bumper smear and raising your insurance rates.
So MiDaS becomes almost magical for artists. When we make a mask in a paint program we use magic wands and bucket fills, while we enlarger everything so big in order to see each pixel we still leave artifacts along the edge of the mask. MiDaS uses AI to automatically generate more accurate masks than we could using simple descriptions of what we want masked. We don't just didn't see the mask as it's display would just slow everything down.
-
Img2img can add detail to an image via depth maps, but it is way more powerful than that. Time to break out paint.net.
I'm going to start another Egyptian themed contest entry. How about a nice caravan of camels crossing the desert? Maybe a few pyramids in the background, and a hot cloudless desert sky. Sounds good; but I don't have a camel figure in my Poser runtime and I'm too slack to model and texture a pyramid in Blender. Why bother? I don't have a camel after all. I do have paint.net. Ok screw it. Paint the top half of a square blue sky. Paint the bottom of the square brown sand. Two yellow triangles for the pyramids. This is going fine. Took less time to paint the background than to type the process. Now for the camels. Three black camels. Beautiful darlings aren't they?
-
Let's just toss this into img2img and tell the AI to do it's thing. I hit it pretty hard. See that Denoising strength slider. That's the "do your own thing" slider. The closer to 0 it is; the more the AI has to listen to what I say. In fact, at 0 the image would not change at all. If I set the slider to 1 I'm saying get as creative as you want. 0.95 is telling the AI I want a lot of creativity in it's interpretation of my lovely camels.
Well damn; this came out better on the first pass than I thought. Camels are facing the wrong way but the rest followed along fairly well. I can live with it. At this point if you wish you can add a depth map from a completely different composition such as a figure posed in a 3D program and a depth map saved from the project. Remember, it doesn't matter the sex, age, or clothing your figure was wearing; we've seen that we can change that at will.
-
Wow, that's unbelievable pdg, just by doing that you got that image, wow, just wow :allhail:
-
I gave the AI almost carte blanche on that one. Usually I would work with much more control over the output by keeping the noise closer to zero and sending the results back over and over again for refinement till I got something closer to my original design. My lovely camels facing the other way.
img2img is just part of the whole pipeline to use in combination with inpainting and outpainting to achieve exactly the image you want. Those two things are up next.
-
Inpainting and outpainting are exactly like they sound. Painting inside an image, much like any paint program; or painting outside the borders of your original image. The first is pretty self explanatory. Take your image from what ever the source and paint adjustments into it via a transparency mask. This is the important part for both inpainting and outpainting. Your paint program must paint nothing where you want it to be transparent. That is one reason I use paint.net. Transparent is just that. Transparent. Some image editors/paint programs use black as a transparency colour. This won't work. Remember our depth masks from earlier. They are made of various amounts of opacity.
So here we go: Let's start with a young Pharaoh for our Egyptian themed contest entry. Here he is in glorious B&W. Now to illustrate a point we will give him striking baby blue eyes. He is a baby after all. We will send our child god image over to inpainting. It could just as easily be a Halloween photo I'm inpainting. Many of the online generation sites will let you inpaint and outpaint with images of your own. They do not need to be AI generated images in order to do this.
-
You can paint out the eyes in your paint program and upload the image. You could upload a mask with the eyes removed to place over the original image. I simply painted vague holes in the original image itself right in my web browser. Tell the AI you want bright blue eyes, or red bloodshot eyes, or green snake eyes. What ever you want. Hit the button and away you go. If you hit the button again it will give different bright blue eyes in the case of our mini Pharaoh.
-
Outpainting is the cool thing. There are various implementations available to try online but the basic idea is the same in all implementations. You get the AI to imagine what your image would look like if the canvas was extended in a given direction. We will take our boy king with the bright blue eyes and drop him in to inpainting/outpainting and ask the AI to extend the canvas upward and fill in the top of our pharaoh's headpiece.Et Viola, our young lad has had a top added to his headpiece. If you don't like it generate another. Changing the denoising will change the result. Outpainting has no size limitations in theory. It most certainly does in practice. The method you chose to do the outpainting and the machine doing the actual outpainting; either CPU or GPU will have RAM, VRAM and or other sizing limitations. If we sent our young contest entrant back thought the outpainting we could finish off each side of the headpiece and maybe bring down the bottom a few rows of pixels to get a more pleasing aspect ratio for our image.
Remember; you can go through img2img, depthmapping, in/outpainting and more over and over again with the help of your paint program refining your composition and details as you go. You can use them in any order you like or not use them at all. The great thing is that all of this stuff can work with your own images and designs. You can ask AI for an image and accept that as the output, but in reality it is more a tool to help with the creativity of the person using it. You will achieve way more artistic satisfaction and understanding of the limitations of the technology once you actually play around with it using AI as a creative tool and part of your workflow instead of letting AI image generation be the end result in and of itself.
-
Because I'm bored and have to wait around here for my lift to arrive; I'll sneak peek something I started working on. I've posted https://youtu.be/R0Duxv2WFnI to the coffee shoppe a while back when I was first exploring this stuff. I was complaining about coherency and how hard it was to attain something passable. That was before depth mapping.
So the way this works is simple:
Use your video tool of choice to take your clip and break it into individual frames. I use the free ffmpeg. It doesn't matter what you use as long as you name the frames with sequential numbers and dump them all into a file folder somewhere.
Take a frame near the start of your video and upload it into the AI. Use the depth mapping model and describe your expected output. Hit generate and examine the output. If the description needs adjusting to get the desired outcome so be it. Adjust the prompt and try again. When you are happy with the image you are generating copy the seed as shown.
-
If we take that seed value and plug it into the AI at a later frame, somewhere near the middle of our video; we can check to be sure all is good or to adjust our prompt if necessary.
We would do that yet again for a frame near the end to quality check. If all looks good just run img2img in batch mode specifying your input images folder and where you want the output images to be placed.
It is then a simple matter of reassembling the frames with ffmpeg, or your editor of choice; back into a mp4, avi, or what have you.
I'll post a link to the output animation when it's done. My lift is here and I'm off.
-
Very interesting read pdg
-
Well I won't show you the video generated. When I got back from dinner all was done. Sadly it is in the state of causing one visual harm with all the jittering. In a hurry to post before I went out yesterday I failed to follow my own advice and check a frame or two near the centre of the video. The little robin took a jump and spin while eating it's seed on the branch. This created a minor freak out for the AI. It did what it was supposed to do; but AI isn't intelligent despite the name and there is actual visual harm and torture to the poor little robin in the machine's output. We can have that. I'd have to start posting those disclaimers about no digital animals being harmed in the creation of the work.
-
:(ROFLMAO: :(ROFLMAO: :(ROFLMAO: :tearlaugh:
-
And they thank you PDG.... :peek: :allhail: :mwah: :madfun: :ty04: :3flower; :ty2: :ty02:
-
This came across my desk this morning as part of my nVidia developer news. Free but you need RTX graphics. Well nVidia; what would one expect. Win 10 and any RTX enabled card will do it as far as I know. https://www.nvidia.com/en-us/studio/canvas/ Should integrate easily enough with whatever other software you use as it will output PSD documents. I used a limited release a year ago and it was pretty good then. This beta update came out for the Consumer Electronics Show in Vegas a week or two ago.
-
This came across my desk this morning as part of my nVidia developer news. Free but you need RTX graphics. Well nVidia; what would one expect. Win 10 and any RTX enabled card will do it as far as I know. https://www.nvidia.com/en-us/studio/canvas/ Should integrate easily enough with whatever other software you use as it will output PSD documents. I used a limited release a year ago and it was pretty good then. This beta update came out for the Consumer Electronics Show in Vegas a week or two ago.
WHOA!!! I have been meaning to explore this. Wasn't sure what it was about. Glad you posted about this, PDG! Now I think I can do my AI artwork offline. Right now, I'm trying to work out an idea for this month's challenge (Egypt). Just saw a movie about the Egyptian gods and that gave me an idea for a scene. So, off I go! :whereizit:
-
You're welcome Hawk. Hope you can use it.
-
I think park they missed another use for AI that would help with their "normal"art. Reverse engineering. Not just using it for postwork. Use it to take a anime/style character over to the more realistic look that can be modeled in Daz or Poser.
-
Generating a 3D object with a clean mesh would be great; but the tools aren't there yet. Soon, but not quite yet. It's the depiction of the unknown backside of the mesh. Generative AI can't "invent" unseen information like the back of a mesh with accuracy. However; there are models being refined ATM that will do such. If you are feeling energetic and willing to suffer disappointment, I can turn you on to such models. I don't recommend them for casual use.
Now; if you want to turn a sketch or drawing into a "realistic" character; not a problem. Something called Controlnet can do this easily using a number of different models. I'll show something later on. It's a little early in the AM to get my motor running, even with a pot of Fafnir's brew.
-
Thanks park. I need more than just his brew today. Doing the windows on the Catholic Church today. Amazed the truck didn't break down.
-
Yeah; well creativity is at a low ATM. I had to call the boys in the white bunny suits twice today. Once when I went to get my Christmas decor from the basement, which I never did get; and then again when I went to take out the recycling. If the waiting list for new housing wasn't longer than my projected lifespan; I'd be gone from here years ago.
In the meantime; take a look at this: https://github.com/lllyasviel/ControlNet
-
Thanks. The breakdown comment comes from the last 3 Easters wanting to go to mass. My truck would break down all three times.
-
Thanks. The breakdown comment comes from the last 3 Easters wanting to go to mass. My truck would break down all three times.
This should be telling you something. :iminnocent:
-
:iminnocent:
-
This came across my desk this morning as part of my nVidia developer news. Free but you need RTX graphics. Well nVidia; what would one expect. Win 10 and any RTX enabled card will do it as far as I know. https://www.nvidia.com/en-us/studio/canvas/ Should integrate easily enough with whatever other software you use as it will output PSD documents. I used a limited release a year ago and it was pretty good then. This beta update came out for the Consumer Electronics Show in Vegas a week or two ago.
Well that puts me out...my graphics card is a paltry GT 520 lol
-
Yeah; I have a 2070 in the small machine and it's not the best for a lot of the newer stuff. You can run generative AI off a CPU these days though. Despite what you may hear in the general media or the business reports, nVidia is under threat from Intel and a few others as the king of the heap. It's only model training that still requires a high powered nVidia card or ten to do.
After all Apple is completely out of the loop when it comes to nVidia cards these days. Intel Macs are gone and the Apple Ski Goggles aren't going to make up for it. Over 5k here for them and you still need a phone or computer to hook them up to. That price buys a new computer with RTX graphics and a new phone to go with it.
Not using RTX graphics I admit is very slow but the times they are a changing. The original post was over a year ago. Things are just cruising along.
-
I am using Dream Studio online...at least that works lol
-
I actually use two. Lol . Pixai and yodayo. Have a couple of others but this pair has the best support that I have found so far. Actually found a pretty good tut on setting up prompts.
-
I will look them up Noshoba...DreamStudio was on the stable diffusion site...
-
I prefer to use one without log in inside. The best I found until now is Stable Diffusion (https://stablediffusionweb.com/fr/WebUI#demo)
Images aren't large, but it's enough for me.
-
My graphics card isn't strong enough to use that one Aelin...I will have to try and save for a new computer!
-
Well I have a decent graphics card. Actually doing most this on my phone apps. Then pop it over to my computer and fiddle with them. My wi-fi connection is my phone hotspot when I need it. Cumbersome but a lot cheaper than paying an extra 130 a month for Internet at home. I am also rarely home as well. So when budget needs pruning it's the first thing to go. Also why I had to give up my modeling program. They went to have to be online to use. So nope...
-
I use a local install of Stable Diffusion amongst other things. Depends what I want to do. I haven't taken a look at it; but I believe the face transfer DAZ currently does is a variation on the ROOP that I use for instance. There are a ton of tools out there. Huggingface that Aelin linked to is the place to go if you want to get an idea of the types of tools and data sets that are out there. It's the de facto repository for machine learning and AI.
-
Well I am not as brainiac as I used to be lol. Biggest annoyance I have is creating multi-person scenes. Now would love to use it for some sci fi modeling but trying to get it to understand is a little rough.
-
Unfortunately PDG, even if I wanted to install an AI on my comp in local, it would not be possible. My little machine is far below from what is aked.
-
Well I am not as brainiac as I used to be lol. Biggest annoyance I have is creating multi-person scenes. Now would love to use it for some sci fi modeling but trying to get it to understand is a little rough.
As I mentioned to Fafnir else where. The current state of generative AI does not allow it to generate the backside of a mesh with any consistency as there are no good 3d weights yet. They are coming soon though.
As for multiple figures might I suggest control net. Do your pose armatures as the control.
-
Unfortunately PDG, even if I wanted to install an AI on my comp in local, it would not be possible. My little machine is far below from what is aked.
There are ways to run Stable Diffusion without a newer graphics card, but it is slow when you use the CPU. Slower than Poser for some things.
-
DreamStudio is working for me at the moment...I don't want to push my computer too far...I only do Firefly renders because trying to use the superfly causes my computer to get a burnt smell...
I did a drawing like your egypt one pdg...I put in the prompt...Futuristic flying spaceships in a new world, moody...
this is what I drew...and the result.
<a href="https://app.photobucket.com/u/Sanbie2/a/1f249294-b22e-45da-8ede-44099df7e20f/p/0c6be36e-1456-46d4-b949-f17195a97cad?width=180&height=180&fit=bounds (https://app.photobucket.com/u/Sanbie2/a/1f249294-b22e-45da-8ede-44099df7e20f/p/0c6be36e-1456-46d4-b949-f17195a97cad?width=180&height=180&fit=bounds)" target="_blank">(https://hosting.photobucket.com/images/g320/Sanbie2/test-01ss_o32b2kH36qB9Zuw1CVNmBv.jpg?width=180&height=180&fit=bounds)
The result...
<a href="https://app.photobucket.com/u/Sanbie2/a/1f249294-b22e-45da-8ede-44099df7e20f/p/a93a6e54-c192-43e6-8028-4ab56579f710?width=180&height=180&fit=bounds (https://app.photobucket.com/u/Sanbie2/a/1f249294-b22e-45da-8ede-44099df7e20f/p/a93a6e54-c192-43e6-8028-4ab56579f710?width=180&height=180&fit=bounds)" target="_blank">(https://hosting.photobucket.com/images/g320/Sanbie2/test-ss.jpg?width=180&height=180&fit=bounds)
-
Not bad Sanbie.
-
Way to go sanbie.
For simple generation try this. https://www.craiyon.com/
If you've a Mickeysoft account, and a lot of us do to access our computers; there is always Bing
-
There's some awesome images on there pdg...will have a proper look tomorrow...gettting tired now...
-
Thought this was interesting.
ChatGPT: The Soul Eater - Nick Cave's Emotional Letter - Read by Stephen Fry
https://www.youtube.com/watch?v=_Pf4GmQY8Ow