FM模型理论之---矩阵形式解读

问题描述

  本文不解释FM模型,仅仅通过向量以及矩阵的形式解释FM模型的理论推导。网络上大部分的推导都是以元素级别的推导,过程显得臃肿。这里将以矩阵和向量的形式解释其偏导数的推导。多项式模型中,特征 x i x_i xi x j x_j xj的组合用 x i x j x_ix_j xixj表示。为了简单起见,我们讨论二阶多项式模型。具体的模型表达式如下(仅讨论二项式部分):
L = Σ i = 1 n − 1   Σ j = i + 1 n w i j ⋅ x i ⋅ x j L= \mathop{\Sigma}\limits_{i=1}^{n-1}\ \mathop{\Sigma}\limits_{j=i+1}^{n}w_{ij}\cdot x_i\cdot x_j L=i=1Σn1 j=i+1Σnwijxixj
  若令 W = V T ⋅ V W=V^T \cdot V W=VTV,其中 V [ k × n ] = [ V 1 , V 2 , ⋯   , V n ] , V i 为 s h a p e 等 于 [ k , 1 ] V_{[k\times n]}=[V_1,V_2,\cdots,V_n],V_i为shape等于[k,1] V[k×n]=[V1,V2,,Vn],Vishape[k,1]的列向量。 w i j w_{ij} wij即为 W [ n × n ] W_{[n \times n]} W[n×n]矩阵的元素。同时令列向量 ξ = [ x 1 , ⋯   , x n ] T \xi = [x_1,\cdots,x_n]^T ξ=[x1,,xn]T。现在我们的目标是寻找 V V V的偏导数?
  注意 w i j w_{ij} wij W W W矩阵的上半区的元素,并且不含有斜对角线上的元素:
∴ L = 1 2 ξ T V T V ξ − 1 2 Σ i = 1 n x i V i T V i x i \therefore L = \frac{1}{2}\xi^T V^T V \xi- \frac{1}{2} \mathop{\Sigma}\limits_{i=1}^{n}x_iV_i^TV_ix_i L=21ξTVTVξ21i=1ΣnxiViTVixi

推导过程

  令:
{ L 1 = 1 2 ξ T V T V ξ L 2 = 1 2 Σ i = 1 n x i V i T V i x i \left\{ \begin{aligned} L_1 &= \frac{1}{2}\xi^T V^T V \xi\\ L_2 &= \frac{1}{2} \mathop{\Sigma}\limits_{i=1}^{n}x_iV_i^TV_ix_i \end{aligned} \right. L1L2=21ξTVTVξ=21i=1ΣnxiViTVixi
则:
∂ L 1 ∂ V = ∂ L 1 ∂ ( V ξ ) ⋅ ∂ V ξ ∂ V = V ξ ⋅ ξ T \begin{aligned} \frac{\partial L_1}{\partial V}&= \frac{\partial L_1}{\partial (V\xi)}\cdot \frac{\partial V\xi}{\partial V}\\ &=V\xi\cdot\xi^T \end{aligned} VL1=(Vξ)L1VVξ=VξξT
同时
∂ L 2 ∂ V i = V i x i 2 \frac{\partial L_2}{\partial V_i}= V_i x_i^2 ViL2=Vixi2
∴ ∂ L 2 ∂ V = [ V 1 x 1 2 , ⋯ V i x i 2 ⋯   , V n x n 2 ] \therefore\frac{\partial L_2}{\partial V}= [V_1 x_1^2,\cdots V_i x_i^2\cdots,V_n x_n^2] VL2=[V1x12,Vixi2,Vnxn2]
  最终结果为:
∴ ∂ L ∂ V = V ξ ⋅ ξ T − [ V 1 x 1 2 , ⋯ V i x i 2 ⋯   , V n x n 2 ] \therefore\frac{\partial L}{\partial V}= V\xi\cdot\xi^T-[V_1 x_1^2,\cdots V_i x_i^2\cdots,V_n x_n^2] VL=VξξT[V1x12,Vixi2,Vnxn2]

总结

   其实一共就仅用了实数对向量求偏导和对矩阵求偏导的知识。本文不做这两个方面的推导,可以参考偏导数链式法则。

后续

  上面的内容是个人能力范围内最简化的推导过程了,但是整个流程下来还是觉得不够过瘾,因为在求 ∂ L 2 / ∂ V {\partial L_2}/{\partial V} L2/V时不免俗的将矩阵 V V V拆开求对向量 V i V_i Vi的偏导,然后再组合为对 V V V的偏导。下面使用矩阵的方法求偏导,不过反而将问题复杂化了(目的还是在于验证直接通过对矩阵求偏导方法的可行性)。
  构造:
X = [ x 1 0 ⋯ 0 0 x 2 ⋯ 0 ⋮ 0 ⋯ 0 x n ] n × n X = \left[ \begin{matrix} x_1 & 0 &\cdots & 0 \\ 0 & x_2 &\cdots & 0 \\ \vdots \\ 0 &\cdots &0 & x_n \end{matrix} \right] _{n\times n} X=x1000x2000xnn×n
∴ L 2 = 1 2 Σ i = 1 n x i V i T V i x i = 1 2 T r ( X V T V X ) ∵ X = X T = 1 2 T r ( X T V T V X ) \begin{aligned} \therefore L_2 &= \frac{1}{2} \mathop{\Sigma}\limits_{i=1}^{n}x_iV_i^TV_ix_i \\ &= \frac{1}{2}Tr(XV^TVX) \qquad \because X=X^T\\ &= \frac{1}{2}Tr(X^TV^TVX) \\ \end{aligned} L2=21i=1ΣnxiViTVixi=21Tr(XVTVX)X=XT=21Tr(XTVTVX)
Z = V ⋅ X Z=V\cdot X Z=VX,即 L 2 = 1 / 2 ⋅ T r ( Z T Z ) L_2 = 1/2\cdot Tr(Z^TZ) L2=1/2Tr(ZTZ)则:
∂ L 2 ∂ Z = Z \frac{\partial L_2}{\partial Z}= Z ZL2=Z
因此:
∂ L 2 ∂ V = ∂ L 2 ∂ Z ⋅ ∂ Z ∂ V = Z ⋅ X T ∵ X = X T = Z ⋅ X = V ⋅ X 2 \begin{aligned} \frac{\partial L_2}{\partial V} &= \frac{\partial L_2}{\partial Z}\cdot \frac{\partial Z}{\partial V}\\ &= Z\cdot X^T \qquad \because X=X^T\\ &= Z\cdot X \\ &= V\cdot X^2 \end{aligned} VL2=ZL2VZ=ZXTX=XT=ZX=VX2
  非常接近结论了, X X X为对角阵,对角线上的元素为 ξ \xi ξ中的变量 x i x_i xi X 2 X^2 X2也为对角阵,用其右乘矩阵 V V V,相当于对矩阵的列向量的乘法操作。
V ⋅ X 2 = [ V 1 , V 2 , ⋯   , V n ] ⋅ X 2 = [ V 1 x 1 2 , ⋯ V i x i 2 ⋯   , V n x n 2 ] V\cdot X^2 = [V_1,V_2,\cdots,V_n]\cdot X^2 = [V_1 x_1^2,\cdots V_i x_i^2\cdots,V_n x_n^2] VX2=[V1,V2,,Vn]X2=[V1x12,Vixi2,Vnxn2]
  可见如果写成矩阵形式:
∴ ∂ L ∂ V = V ξ ⋅ ξ T − V ⋅ X 2 \therefore\frac{\partial L}{\partial V}= V\xi\cdot\xi^T- V\cdot X^2 VL=VξξTVX2

你可能感兴趣的:(FM模型理论之---矩阵形式解读)