win10部署NovelAI绘画，Stable Diffusion，Chilloutmix，实现txt2img，img2img（含调参）

1、Stable Diffusion是什么

• Stable Diffusion是2022年发布的深度学习文本到图像生成模型。它主要用于根据文本的描述产生详细图像，尽管它也可以应用于其他任务，如内补绘制、外补绘制，以及在提示词​（英语）指导下产生图生图的翻译。
• Stable Diffusion是由德国慕尼黑大学机器视觉与学习研究小组和Runway的研究人员基于CVPR2022的一篇论文：《High-Resolution Image Synthesis with Latent Diffusion Models》，并与其他社区团队合作开发的一款开源模型。

2、深度学习环境搭建

1. Anaconda下载与安装：地址
2. Pycharm（IDE）下载与安装：地址
3. CUDA、CuDNN下载与安装：地址

3、Stable Diffusion环境搭建（可跳过）

参考资料：地址

程序下载：地址
模型下载：地址

Github下载项目并打开运行，缺一小部分库。

打开终端，安装相关库
pytorch_lightning, omegaconf, einops

或者使用conda创建虚拟环境一键安装依赖
conda env create -f environment.yaml
conda activate ldm

下载权重
地址
在项目目录models/ldm下手动创建stable-diffusion-v1文件夹，用来存放下载好的权重文件，即需要把下载文件夹里的sd-v1-4.ckpt文件改名为model.ckpt，然后粘贴到手动创建的stable-diffusion-v1文件夹下即可

遇到报错NameError: name ‘trainer’ is not defined. Did you mean: ‘Trainer’?
根据官方wiki的教程进行修补 地址

报错2
地址

4、Stable Diffusion WebUI环境搭建（主要）

Github下载star最多的stable-diffusion-webui项目（自带图形化包）
地址

• 使用conda创建python.3.10.6虚拟环境
  conda create -n stable-webui python=3.10.6
  

• 激活虚拟环境
  conda activate stable-webui
  

• 进入刚刚git clone的目录，运行webui-user.bat
  等待自动安装进度条
  

• 根据提示增加一些git配置信息
  git config --global --add safe.directory xxx/stable-diffusion-webui/repositories/taming-transformers
  

• 修改配置文件
  增加—share解决本地localhost权限问题
  

• 继续执行webui-user.bat，成功启动服务
  

• 改一下参数，网上复制一段描述性的话，使用默认模型随机生成几张图。
  

5、NovelAI模型的修改与调参

模型可以在网上资源贴中下载合适的模型（.ckpt），放到models\Stable-diffusion里。
比如：地址

这里我们选择NovelAI泄漏的完整版本，用于二次元绘画
使用刚才的参数直接训练一下

描述词

{anime screenshot}(masterpiece, best quality, illustration, cinematic lighting，CG，wallpaper)
1 girl , beautifully detailed eyes，messy hair，long hair, flipped hair，Hair glows，pov，
blush，White hair，Red eyes，animal ears， looking at viewer，collar，dramatic angle,

(Intense POV) Highly detailed still professional portrait of anime film (a cute kawaii girl) destroyed in a tunic on a balcony in the city (overgrown with stuff), focus on the shoulders from above Pale skin on white marble glow very beautiful blue eyes pupil iris blush soft ambient light lens flare light particles beautiful composition window

排除词

multiple breasts, (mutated hands and fingers:1.5 ), (long body :1.3), (mutation, poorly drawn :1.2) , black-white, bad anatomy, liquid body, liquid tongue, disfigured, malformed, mutated, anatomical nonsense, text font ui, error, malformed hands, long neck, blurred, lowers, lowres, bad anatomy, bad proportions, bad shadow, uncoordinated body, unnatural body, fused breasts, bad breasts, huge breasts, poorly drawn breasts, extra breasts, liquid breasts, heavy breasts, missing breasts, huge haunch, huge thighs, huge calf, bad hands, fused hand, missing hand, disappearing arms, disappearing thigh, disappearing calf, disappearing legs, fused ears, bad ears, poorly drawn ears, extra ears, liquid ears, heavy ears, missing ears, fused animal ears, bad animal ears, poorly drawn animal ears, extra animal ears, liquid animal ears, heavy animal ears, missing animal ears, text, ui, error, missing fingers, missing limb, fused fingers, one hand with more than 5 fingers, one hand with less than 5 fingers, one hand with more than 5 digit, one hand with less than 5 digit, extra digit, fewer digits, fused digit, missing digit, bad digit, liquid digit, colorful tongue, black tongue, cropped, watermark, username, blurry, JPEG artifacts, signature, 3D, 3D game, 3D game scene, 3D character, malformed feet, extra feet, bad feet, poorly drawn feet, fused feet, missing feet, extra shoes, bad shoes, fused shoes, more than two shoes, poorly drawn shoes, bad gloves, poorly drawn gloves, fused gloves, bad cum, poorly drawn cum, fused cum, bad hairs, poorly drawn hairs, fused hairs, big muscles, ugly, bad face, fused face, poorly drawn face, cloned face, big face, long face, bad eyes, fused eyes poorly drawn eyes, extra eyes, malformed limbs, more than 2 nipples, missing nipples, different nipples, fused nipples, bad nipples, poorly drawn nipples, black nipples, colorful nipples, gross proportions. short arm, (((missing arms))), missing thighs, missing calf, missing legs, mutation, duplicate, morbid, mutilated, poorly drawn hands, more than 1 left hand, more than 1 right hand, deformed, (blurry), disfigured, missing legs, extra arms, extra thighs, more than 2 thighs, extra calf, fused calf, extra legs, bad knee, extra knee, more than 2 legs, bad tails, bad mouth, fused mouth, poorly drawn mouth, bad tongue, tongue within mouth, too long tongue, black tongue, big mouth, cracked mouth, bad mouth, dirty face, dirty teeth, dirty pantie, fused pantie, poorly drawn pantie, fused cloth, poorly drawn cloth, bad pantie, yellow teeth, thick lips, bad cameltoe, colorful cameltoe, bad asshole, poorly drawn asshole, fused asshole, missing asshole, bad anus, bad pussy, bad crotch, bad crotch seam, fused anus, fused pussy, fused anus, fused crotch, poorly drawn crotch, fused seam, poorly drawn anus, poorly drawn pussy, poorly drawn crotch, poorly drawn crotch seam, bad thigh gap, missing thigh gap, fused thigh gap, liquid thigh gap, poorly drawn thigh gap, poorly drawn anus, bad collarbone, fused collarbone, missing collarbone, liquid collarbone, strong girl, obesity, worst quality, low quality, normal quality, liquid tentacles, bad tentacles, poorly drawn tentacles, split tentacles, fused tentacles, missing clit, bad clit, fused clit, colorful clit, black clit, liquid clit, QR code, bar code, censored, safety panties, safety knickers, beard, furry ,pony, pubic hair, mosaic, excrement, faeces, shit

• 然后是img2img，我们增加结果与原图的不同程度，得到如下
  

6、Chilloutmix-Ni模型

模型寻找攻略：1
2

• 然后我们再换用常用的3D模型生成一些图
  
  

• 最后我们使用真人模型Chilloutmix-Ni进行训练