CTR 预测理论（二十五）：矩阵和向量乘法总结
推荐系统中常涉及矩阵、向量乘法，此处结合现有文献做一个小结，仅用于学习交流使用。
1. 矩阵乘法
矩阵乘积(matrix product，也叫matmul product)： A m × n ⋅ B n × p = C m × p A_{m\times n} \cdot B_{n \times p} = C_{m \times p} Am×n⋅Bn×p=Cm×p ， A A A 的列数必须和 B B B 的行数相等， C i , j = ∑ k = 1 n A i , k B k , j C_{i, j}=\sum_{k=1}^{n} A_{i, k} B_{k, j} Ci,j=∑k=1nAi,kBk,j。
Hadamard product(又称element-wise product)：用 ⊙ \odot ⊙ 或者 ∘ \circ ∘ 表示， A ⊙ B A \odot B A⊙B 对应元素相乘，二者维数必须相同，举例：
[ a 11 a 12 a 21 a 22 ] ⊙ [ b 11 b 12 b 21 b 22 ] = [ a 11 b 11 a 12 b 12 a 21 b 21 a 22 b 22 ] \left[\begin{array}{ll}a_{11} & a_{12} \\a_{21} & a_{22}\end{array}\right] \odot\left[\begin{array}{ll}b_{11} & b_{12} \\b_{21} & b_{22}\end{array}\right]=\left[\begin{array}{ll}a_{11} b_{11} & a_{12} b_{12} \\a_{21} b_{21} & a_{22} b_{22}\end{array}\right] [a11a21a12a22]⊙[b11b21b12b22]=[a11b11a21b21a12b12a22b22]
Kronecker product(克罗内克积)：用 ⊗ \otimes ⊗ 表示，两个任意大小矩阵间的运算， A m × n ⋅ B p × q = C m p × n q A_{m \times n} \cdot B_{p \times q}=C_{m p \times n q} Am×n⋅Bp×q=Cmp×nq， A A A 的每个元素逐个与矩阵 B B B 相乘。
[ a 11 a 12 a 21 a 22 ] ⊗ [ b 11 b 12 b 21 b 22 ] = [ a 11 b 11 a 11 b 12 a 12 b 11 a 12 b 12 a 11 b 21 a 11 b 22 a 12 b 21 a 12 b 22 a 21 b 11 a 21 b 12 a 22 b 11 a 22 b 12 a 21 b 21 a 21 b 22 a 22 b 21 a 22 b 22 ] \left[\begin{array}{cc}a_{11} & a_{12} \\a_{21} & a_{22}\end{array}\right] \otimes\left[\begin{array}{cc}b_{11} & b_{12} \\b_{21} & b_{22}\end{array}\right]=\left[\begin{array}{cccc}a_{11} b_{11} & a_{11} b_{12} & a_{12} b_{11} & a_{12} b_{12} \\a_{11} b_{21} & a_{11} b_{22} & a_{12} b_{21} & a_{12} b_{22} \\a_{21} b_{11} & a_{21} b_{12} & a_{22} b_{11} & a_{22} b_{12} \\a_{21} b_{21} & a_{21} b_{22} & a_{22} b_{21} & a_{22} b_{22}\end{array}\right] [a11a21a12a22]⊗[b11b21b12b22]=⎣⎢⎢⎡a11b11a11b21a21b11a21b21a11b12a11b22a21b12a21b22a12b11a12b21a22b11a22b21a12b12a12b22a22b12a22b22⎦⎥⎥⎤
矩阵乘积的常用性质：
结合律： A ( B + C ) = A B + A C A(B+C) = AB + AC A(B+C)=AB+AC
分配律： A ( B C ) = ( A B ) C A(BC)=(AB)C A(BC)=(AB)C
交换律：绝大多数情况不满足交换律，即大多数情况下 A B ≠ B A AB \ne BA AB=BA 。
2. 向量乘法
向量点积/内积(Inner Product, dot product)：用 ⋅ \cdot ⋅ 表示，两个向量的行列数必须相同，点乘的结果是对应位元素相乘后求和，是一个标量，举例：
a = ( a 1 , a 2 , ⋯ , a n ) b = ( b 1 , b 2 , ⋯ , b n ) a ⋅ b = a 1 b 1 + a 2 b 2 + ⋯ + a n b n \begin{array}{c}a=\left(a_{1}, a_{2}, \cdots, a_{n}\right) \\b=\left(b_{1}, b_{2}, \cdots, b_{n}\right) \\a \cdot b=a_{1} b_{1}+a_{2} b_{2}+\cdots+a_{n} b_{n}\end{array} a=(a1,a2,⋯,an)b=(b1,b2,⋯,bn)a⋅b=a1b1+a2b2+⋯+anbn
点乘的几何意义：点乘可以用来计算两个向量的夹角 c o s θ = a ⋅ b ∣ a ∣ ∣ b ∣ cos \theta = \frac{a \cdot b}{|a||b|} cosθ=∣a∣∣b∣a⋅b。
dot product 和 inner product 其实还是有区别的，目前暂时将二者视为同一个概念，后续再来细究！！！
向量外积(Outer product)：外积的结果是一个矩阵，用 ⊗ \otimes ⊗ 表示，举例；
u = ( u 1 , u 2 , ⋯ , u m ) v = ( v 1 , v 2 , ⋯ , v n ) u ⊗ v = [ u 1 v 1 u 1 v 2 ⋯ u 1 v n u 2 v 1 u 2 v 2 ⋯ u 2 v n ⋮ ⋮ ⋱ ⋮ u m v 1 u m v 2 ⋯ u m v n ] \begin{array}{c}u=\left(u_{1}, u_{2}, \cdots, u_{m}\right) \\v=\left(v_{1}, v_{2}, \cdots, v_{n}\right) \\u \otimes v=\left[\begin{array}{ccccc}u_{1} v_{1} & u_{1} v_{2} & \cdots & u_{1} v_{n} \\u_{2} v_{1} & u_{2} v_{2} & \cdots & u_{2} v_{n} \\\vdots & \vdots & \ddots & \vdots \\u_{m} v_{1} & u_{m} v_{2} & \cdots & u_{m} v_{n}\end{array}\right]\end{array} u=(u1,u2,⋯,um)v=(v1,v2,⋯,vn)u⊗v=⎣⎢⎢⎢⎡u1v1u2v1⋮umv1u1v2u2v2⋮umv2⋯⋯⋱⋯u1vnu2vn⋮umvn⎦⎥⎥⎥⎤
向量叉积(Cross product)：叉乘的结果是一个向量，使用符号 × \times ×，举例：
a = ( x 1 , y 1 , z 1 ) b = ( x 2 , y 2 , z 2 ) a × b = ∣ i j k x 1 y 1 z 1 x 2 y 2 z 2 ∣ = ( y 1 z 2 − y 2 z 1 ) i + ( z 1 x 2 − z 2 x 1 ) j + ( x 1 y 2 − x 2 y 1 ) k i = [ 1 , 0 , 0 ] , j = [ 0 , 1 , 0 ] , k = [ 0 , 0 , 1 ] \begin{aligned}&a=\left(x_{1}, y_{1}, z_{1}\right)\\&b=\left(x_{2}, y_{2}, z_{2}\right)\\&a \times b=\left|\begin{array}{ccc}i & j & k \\x_{1} & y_{1} & z_{1} \\x_{2} & y_{2} & z_{2}\end{array}\right|=\left(y_{1} z_{2}-y_{2} z_{1}\right) i+\left(z_{1} x_{2}-z_{2} x_{1}\right) j+\left(x_{1} y_{2}-x_{2} y_{1}\right) k\\&i=[1,0,0], j=[0,1,0], k=[0,0,1]\end{aligned} a=(x1,y1,z1)b=(x2,y2,z2)a×b=∣∣∣∣∣∣ix1x2jy1y2kz1z2∣∣∣∣∣∣=(y1z2−y2z1)i+(z1x2−z2x1)j+(x1y2−x2y1)ki=[1,0,0],j=[0,1,0],k=[0,0,1]
叉乘的几何意义：向量叉乘的结果是两个向量的法向量，举个例子： a a a 是 x x x 轴的单位向量， b b b 是 y y y 轴的单位向量，二者叉乘的结果就是 z z z 轴的单位向量。
a = ( 1 , 0 , 0 ) b = ( 0 , 1 , 0 ) i = ( 1 , 0 , 0 ) j = ( 0 , 1 , 0 ) k = ( 0 , 0 , 1 ) a × b = ∣ i j k 1 0 0 0 1 0 ∣ = ( 0 × 0 − 0 × 1 ) i + ( 0 × 0 − 0 × 1 ) j + ( 1 × 1 − 0 × 0 ) k = k \begin{array}{c}a=(1,0,0) \\b=(0,1,0) \\i=(1,0,0) \\j=(0,1,0) \\k=(0,0,1) \\a \times b=\left|\begin{array}{ccc}i & j & k \\1 & 0 & 0 \\0 & 1 & 0\end{array}\right|=(0 \times 0-0 \times 1) i+(0 \times 0-0 \times 1) j+(1 \times 1-0 \times 0) k=k\end{array} a=(1,0,0)b=(0,1,0)i=(1,0,0)j=(0,1,0)k=(0,0,1)a×b=∣∣∣∣∣∣i10j01k00∣∣∣∣∣∣=(0×0−0×1)i+(0×0−0×1)j+(1×1−0×0)k=k
3. 总结
| 矩阵乘法 | 符号 | 说明 |
|---|---|---|
| matmul product | ⋅ \cdot ⋅ | 一般的矩阵乘积 |
| Hadamard product | ⊙ \odot ⊙ 或者 ∘ \circ ∘ 表示 | element-wise，逐元素对应乘积 |
| Kronecker product (克罗尔克积) | ⊗ \otimes ⊗ | 任意形状矩阵相乘 |
| 向量乘法 | 符号 | 说明 |
|---|---|---|
| Inner product, dot product | ⋅ \cdot ⋅ | 结果是标量 |
| Outer product | ⊗ \otimes ⊗ | 结果是矩阵 |
| Cross product | × \times × | 结果是向量 |
参考文献：
[1] https://zhuanlan.zhihu.com/p/79760117