【LLM】大模型中的温度系数是啥玩意

一、LLM中的温度系数

• temperature参数控制生成语言模型中生成文本的随机性和创造性，调整模型的softmax输出层中预测词的概率；
• 其值越大，则预测词的概率的方差减小，即很多词被选择的可能性增大，利于文本多样化
• 举例：Prompt: “The quick brown fox”
  • Temperature = 0.1:
    “The quick brown fox jumped over the lazy dog. The quick brown fox jumped over the lazy dog. The quick brown fox jumped over the lazy dog.
  • Temperature = 0.5:
    “The quick brown fox jumped over the lazy dog. The lazy cat was not impressed. The quick brown fox ran away.”
  • Temperature = 1.0:
    “The quick brown fox jumped over the lazy dog. Suddenly, a flock of birds flew overhead, causing the fox to stop in its tracks. It looked up at the sky, wondering where they were going.”

softmax计算的概率分布：$$p\left(x_i\right)=\frac{e^{x_i}}{{\sum }_{j=1}^V{e}^{x_j}}$$
加入温度系数到softmax时：$$\begin{gathered} p\left(x_i\right)=\frac{e^{\frac{x_i}{T}}}{{\sum }_{j=1}^V{e}^{\frac{x_j}{T}}} \\ =\frac{1}{{\sum }_{j=1}^V{e}^{\frac{x_j-x_i}{T}}} \end{gathered}$$
简单分析：temperature从0到1的数值，当数值越大，将上式的分子分母同时除以分子，就可以发现temperture越大则总体值越大。

二、代码举例

import torch
import torch.nn as nn
import torch.nn.functional as F
import matplotlib.pyplot as plt

criterion = nn.CrossEntropyLoss()
x = torch.Tensor([[1, 2, 3 ,4]])
y = torch.LongTensor([3])
# 折线图横坐标
x_ = [i for i in range(len(x.squeeze(0)))]
t = [1, 0.5, 0.1]
col = ['red', 'blue', 'green']

for i in range(len(t)):
    temp = t[i]
    out = F.softmax(x / temp, dim=1)
    loss = criterion(out, y)
    y_ = out.squeeze(0).tolist()
    plt.plot(x_, y_, color=col[i], linestyle='dashed', linewidth=2, \
             marker='o', markerfacecolor='red', markersize=8, label = "temp:" + str(temp))
    print("softmax概率值：", out, "\nloss:", loss, "\n")

# 图例放在右上角
plt.legend(loc = "upper left")
plt.title("diffterent temperature")
plt.xlabel("xlabel")
plt.ylabel("logits")
plt.show()

随着T的减少，softmax输出各类别的概率方差越大，而T越大则越相对平滑，印证了刚才的分析。

Reference

[1] 谈谈softmax中常出现的温度系数 T (τ)
[2] LLM Parameters Demystified: Getting The Best Outputs from Language AI