stable diffusion实践操作-常见lora模型介绍

本文专门开一节写Lora相关的内容，在看之前，可以同步关注：
stable diffusion实践操作

模型分两种，一种是sd大模型，一种是类似Lora的小模型
国内的是：https://www.liblibai.com
国外的是：https://civitai.com

一、什么是lora?

1.1 lora 定义

LoRA，英文全称Low-Rank Adaptation of Large Language Models，直译为大语言模型的低阶适应，这是微软的研究人员为了解决大语言模型微调而开发的一项技术。

比如，GPT-3有1750亿参数，为了让它能干特定领域的活儿，需要做微调，但是如果直接对GPT-3做微调，成本太高太麻烦了。

LoRA的做法是，冻结预训练好的模型权重参数，然后在每个Transformer（Transforme就是GPT的那个T）块里注入可训练的层，由于不需要对模型的权重参数重新计算梯度，所以，大大减少了需要训练的计算量。

研究发现，LoRA的微调质量与全模型微调相当，我愿称之为神器。

要做个比喻的话，就好比是大模型的一个小模型，或者说是一个插件。

LoRA本来是给大语言模型准备的，但把它用在cross-attention layers（交叉关注层）也能影响用文字生成图片的效果。

1.2 lora的基本原理

（把lora里面的十几个层风别抽出来，单独设置权重）
这一块，慢慢补充。

二、作用：

2.1 复刻人物特征

对人物和物品的复刻，只要挂载lora，就可以九成九的复刻指定人物的特征。
比如，你下载了光头强的Lora，只要你加载了它，那么你生成的任何一张图片都是他的样子。

2.2 固定人物的动作特征

2.3 确定某种画风

总结：Lora可以确定某种特征。

Lora一定要和底模配合使用。

三、lora和embedding 区别：

3.1 文件大小不一样

文件小携带的信息就比较少。

3.2 lora比embedding要好的多

embedding在多视图和动漫人物上要好一点。

3.3 embedding是文字描述，lora是真实的物品描述。

四 、lora使用

用lora配套的大模型

因为lora训练是需要一个大模型的，是通过大模型来训练参数的人，所以它的一切和大模型都绑定了。
所以使用的时候，要和训练的原始模型匹配。
下面是hanfu lora和不同大模型的对比：

有些效果可以，有些效果不好，这就开盲盒了。

4.2 .有触发词一定要使用触发词

<lora:baihuaniang_1.0:0.8>,baihuaniang，flower

一般来说，不用把全部的触发词加上去，只要加其中一个就可以了，比如下面的有汉服，有唐服等等，这个要根据lora模型来确定，有写lora是没有提示词的。

4.3 正确设置lora的权重

权重越大，lora对结果影响就很大。但是也不能过大，过小，一般在0.6-0.9之间。

4.4 新手不要混用lora

多了会打架。

正文

---

序号	lora名称	说明
1	baihuaniang_1.0
2	long	龙
3	xiaorenshu_v2.0	小人书
4	Lora_GuoFeng3_Lora_v3.2_Lora_big_light	中国风
5	Real dragons _dragon_real_base_V1	real dragon
6	korean doll likeness	美超真实女
7	Beautiful_Dress	连衣裙
8	$1
9	$1
10	$1

１、baihuaniang_1.0

下载链接：https://www.liblibai.com/modelinfo/1d2839ecaaf829e0161beb8c70294dbf
基础模型：SD1.5（AWPainting_v1.2_liblib_sd15.safetensors）
触发词语：baihuaniang，flower

案例：
正向提示词：

Best quality,masterpiece,ultra high res,(photorealistic:1.4),
solo,1girl,.<lora:baihuaniang_1.0:0.8>,baihuaniang,Flowers,
Flowers,Soft sunlight,Wind,Flying long hair,Film Light,
Facing the screen,.copper,,upper body,fatigues,
outstretched arms,.

反向提示词：

ng_deepnegative_v1_75t,badhandv4 (worst quality:2),
(low quality:2),(normal quality:2),lowres,bad anatomy,
bad hands,normal quality,((monochrome)),((grayscale)),, 
ng_deepnegative_v1_75t, badhandv4 (worst quality:2), 
(low quality:2), (normal quality:2), lowres, bad anatomy, 
bad hands, normal quality, ((monochrome)), ((grayscale)):.

采样：
DPM++2MKarras
随机种子：2514900086
CFGscale:11
步数：28

生成结果：

2、long.safetensors　中国龙

下载链接：https://www.liblibai.com/modelinfo/870bee7ee8377e037097ae2d16839709
触发词：no humans， dragon， long
基础模型SD1.5 （AWPainting_v1.2_liblib_sd15.safetensors）
正向提示词：

Best quality,masterpiece,ultra high res,nu no humans, (long:1.2),
<lora:long:0.8>,no humans， dragon， long eastern dragon, 
east asian architecture, sky, architecture, outdoors, cloud, day, 
mountain, open mouth, blue sky, teeth, scales, sharp teeth, 
horns, glowing, fangs, tree, glowing eyes, pagoda, scenery.

反向提示词：

ng_deepnegative_v1_75t,badhandv4 (worst quality:2),(low quality:2),
(normal quality:2),lowres,bad anatomy,bad hands,normal quality,
((monochrome)),((grayscale)),nsfw,..

采样：
DPM++2MKarras
随机种子：2330800025
CFGscale:11
步数：28

生成结果：

3、xiaorenshu_v2.0.safetensors

下载链接：https://www.liblibai.com/modelinfo/7755bdd92e3baeb7bd8ac79a455070ad
基础模型：SD1.5 （AWPainting_v1.2_liblib_sd15.safetensors）
触发词语：没有

正向提示词：
shz,((bestquality)),((masterpiece)),(detailed),realistic,multipleboys,(ridinghorse),mustache,lookingatviewer,outdoors,house,amy,horse

反向提示词：

采样：DPM++2MKarras
随机种子：3836055402
CFGscale: 5
步数：28

生成结果：

4、Lora_GuoFeng3_Lora_v3.2_Lora_big_light.safetensors

下载链接：https://www.liblibai.com/modelinfo/7b3ce08ce87271586f0fd00de32604be
基础模型：SD1.5 （AWPainting_v1.2_liblib_sd15.safetensors）
触发词语：没有

正向提示词：

bestquality,masterpiece,highres,1girl,chinadress,Beautifulface,
hairornament,solo,lookingatviewer,smile,closedmouth,lips,
chinadress,dress,hairornament,necklace,jewelry,longhair,earrings,chineseclothes,
architecture,eastasianarchitecture,building,outdoors,rooftop,city,cityscape,
darkstudio,rimlighting,twotonelighting,dimlylit,lowkey
<lora:GuoFeng3.2_Lora_light:1.5>

采样：Eulera
随机种子：3985684671
CFGscale: 7
步数：30

生成结果：

5、Lora Real dragons _dragon_real_base_V1.safetensors

下载链接：https://www.liblibai.com/modelinfo/6d1bbd25afd9c54f442606d1774dc425
基础模型：SD1.5 （AWPainting_v1.2_liblib_sd15.safetensors）
触发词语：dragon_real，flame

正向提示词：

dragon_real,(extremely detailed CG unity 8k wallpaper,masterpiece, best quality, 
ultra-detailed, beautiful detailed eyes:1.2),best illumination, (best shadow, an extremely delicate and beautiful, bloom),
the girl was surrounded by dragons,.(1girl:1.4),breasts, solo,full body,Back tattoo,white dress,(color dragon:1.4),
Bare leg,long dress, 

.
反向提示词：

nude, (badhandv4:1.2), (easynegative), verybadimagenegative_v1.3, deformation), blurry, poorly dissected, disfigured,
 poorly drawn face, mutation, mutation, (extra_limb), (ugly), (poorly drawn hand), finger fusion, messy drawing, 
broken leg examination, examination, examination stick, multiple breasts, (mutated hand and finger: 1.5), (long body: 1.3), 
(mutated, poorly drawn: 1.2), black and white, poorly dissected, liquid body, disfigured Deformed, mutated, anatomical nonsense, 
text font ui, errors, malformed hands, long necks, blurry, lowered, anatomically poor, proportion not good, shadows not good, 
body not coordinated, body not natural, fused breasts, poor breasts, huge breasts, poorly drawn breasts, excess breasts, 
liquid breasts, heavy breasts, missing breasts, large buttocks, huge thighs, Huge calves, bad hands, fused hands, missing hands, 
missing arms, missing thighs, missing calves, missing legs, fused arms, bad ears, poorly drawn ears, excess ears, liquid ears, 
heavy ears, missing ears, text, ui, errors, missing fingers, missing limbs, fused fingers, one hand with more than 5 fingers, 
one hand with less than 5 fingers, The other hand has more than 5 digits, the other hand is missing less than 5 digits, 
excess digits, fewer digits, fused digits, missing digits, bad digits, liquid digits, cropped, blurry, JPEG artifacts, signatures, 
deformed feet, excess feet, bad feet, bad drawings, fused feet, lost feet, excess shoes, bad shoes, fused shoes,
 two or more pairs of shoes, poorly drawn shoes Bad gloves, poorly drawn gloves, fused gloves, bad hair, poorly drawn hair, 
fused hair, big muscles, ugly, bad face, fused face, poorly drawn face, cloned face, big face, long face, bad eyes, 
fused eyes well drawn eyes, excess eyes, deformed limbs, two or more nipples, rough ratio. Short arm, (((missing arm)), 
thigh missing, calf missing, leg missing, mutation, duplication, deformity, poorly drawn hand, more than one left hand, 
less than one right hand, deformation, (blurring), disfigurement, missing leg, excess arm, excess thigh, two or more thighs, 
excess calf, fused calf, excess leg, poor knee, excess knee, more than two legs, Bad mouth, fused mouth, bad mouth, 
bad tongue, tongue in mouth, too long tongue, black tongue, big mouth, cracked mouth, bad mouth, dirty face, 
dirty teeth, bad underpants, fused cloth, bad cloth, bad underpants, yellow teeth, thick lips, bad camel toe, 
colorful camel toe, bad crotch seam, missing clip, bad clip, fused clip, color clip, black clip, liquid clip, QR code,
 barcode, reviewed, secure underwear, extra numbers, fewer numbers, cropped, worst quality, low quality, 
normal quality, jpeg artifacts, signature, watermark, username, blurry, out of focus, reviewed, ugly, old, deformed, 
amateur, strange, fat, deformed, black and white, realistic eyes,.

采样：DPM++2MKarras
随机种子：2656310671
CFGscale: 7
步数：55

生成结果：

6、korean doll likeness

下载链接：https://www.aigc.wtf/korean-doll-likeness-free-download/
成图参考：https://zhuanlan.zhihu.com/p/606825889
基础模型：chilloutmix_NiPrunedFp16Fix.safetensors
触发词语：

正向提示词：

(RAW photo:1.2), (photorealistic:1.4),(masterpiece:1.3),(best quality:1.4),
nikon d750 f/1.4 55mm, detailed shadow, (realistic, photo-realistic:1.37),(sharp focus:1.2),(ISO high), (masterpiece,best quality), professional lighting, ultra high res,

dreamlike,
(detailed eyes),(detailed facial features), (detailed clothes features),
skin tight, (shiny skin),(slender girl),(slim girl),
1girl, ((full body)),solo,
cute,smlie,
, (medium breasts), bangs, straight hair, long hair, black hair, black eyes, crystal pendant,
Long blue scarf,White long wool coat,Black thick pantyhose,Brown snow boots,
<lora:koreanDollLikeness_v15:1>

.
反向提示词：

nsfw,EasyNegative,bad_prompt,ng_deepnegative_v1_75t,(worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality, ((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot, (outdoor:1.6), manboobs, backlight,(ugly:1.331), (duplicate:1.331), (morbid:1.21), (mutilated:1.21), (tranny:1.331), mutated hands, (poorly drawn hands:1.331), blurry, (bad anatomy:1.21), (bad proportions:1.331), extra limbs, (disfigured:1.331), (more than 2 nipples:1.331), (missing arms:1.331), (extra legs:1.331), (fused fingers:1.61051), (too many fingers:1.61051), (unclear eyes:1.331), bad hands, missing fingers, extra digit, (futa:1.1), bad body, pubic hair, glans,
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 546686100, Size: 720x960, Model hash: fc2511737a, Model: chilloutmix_NiPrunedFp32Fix (2), Clip skip: 2, ENSD: 31337

采样：DPM++2MKarras
随机种子：2514900086
CFGscale: 7
步数：22
生成结果：

7、Beautiful_Dress连衣裙

下载链接：https://www.liblibai.com/modelinfo/3aa33b228cda4a0980754ae921eb51d4
成图参考：
基础模型：chilloutmix_NiPrunedFp16Fix.safetensors
触发词语：dress
说明：这个只会生成裙子，一般没有人头，最好和其它的配合使用

正向提示词：

<lora:Beautiful_Dress:0.6>,(tightlacedressblackdeepvnecklayeredskirtankletdecorateholdflowersultryversatile:1.1),Girl,lightblue,(fullbody:1.2),(wholebody:1.2),(Lightweight:1.1),(Smile:1.1),Legs,Pure,Feet,(Lightmakeup:1.1),(45°LookUp:1.1),blackhair,Longhair,(whiteground:1.2),(Lightbackground:1.2),LongDress,(Stand.:1.2),Silkytexture,Cool,(Summer:1.1),(masterpiece:1,2),bestquality,masterpiece,highres,original,extremelydetailedwallpaper,perfectlighting,(extremelydetailedCG:1.2),drawing,paintbrush,Freedom,Romantic,mediumbreasts,

.
反向提示词：

NSFW,(worst quality:2),(low quality:2),(normal quality:2),lowres,normal quality,((monochrome)),((grayscale)),skin spots,acnes,skin blemishes,age spot,(ugly:1.331),(duplicate:1.331),(morbid:1.21),(mutilated:1.21),(tranny:1.331),mutated hands,(poorly drawn hands:1.5),blurry,(bad anatomy:1.21),(bad proportions:1.331),extra limbs,(disfigured:1.331),(missing arms:1.331),(extra legs:1.331),(fused fingers:1.61051),(too many fingers:1.61051),(unclear eyes:1.331),lowers,bad hands,missing fingers,extra digit,bad hands,missing fingers,(((extra arms and legs))),

采样：DPM++2MKarras
随机种子：1172117708

CFGscale: 7
步数：22

生成结果：