矩阵求导与Hessian矩阵

  • 标量关于标量的导数

\frac{dy}{dx}

 

  • 向量关于标量的导数

设向量矩阵求导与Hessian矩阵_第1张图片和标量x,

矩阵求导与Hessian矩阵_第2张图片

 

  • 矩阵关于标量的导数

设M×N矩阵矩阵求导与Hessian矩阵_第3张图片和标量x,

\frac{dY}{dx}=\begin{bmatrix} \frac{dy_{11}}{dx} &\frac{dy_{12}}{dx} &... &\frac{dy_{1n}}{dx} \\ \frac{dy_{21}}{dx} &\frac{dy_{22}}{dx} &... &\frac{dy_{2n}}{dx} \\ ... &... &... &... \\ \frac{dy_{m1}}{dx} &\frac{dy_{m2}}{dx} &... &\frac{dy_{mn}}{dx} \end{bmatrix}

 

  • 标量关于向量的导数

设标量y和向量矩阵求导与Hessian矩阵_第4张图片

矩阵求导与Hessian矩阵_第5张图片

 

  • 向量关于向量的导数

设向量矩阵求导与Hessian矩阵_第6张图片和向量x=\begin{bmatrix} x_{1}& x_{2}& ...& x_{n} \end{bmatrix}

\frac{dy}{dx}=\begin{bmatrix} \frac{\partial y_{1}}{\partial x_{1}} &\frac{\partial y_{1}}{\partial x_{2}} &... &\frac{\partial y_{1}}{\partial x_{n}} \\ \frac{\partial y_{2}}{\partial x_{1}} &\frac{\partial y_{2}}{\partial x_{2}} &... &\frac{\partial y_{2}}{\partial x_{n}} \\ ... &... &... &... \\ \frac{\partial y_{m}}{\partial x_{1}} &\frac{\partial y_{m}}{\partial x_{2}} &... &\frac{\partial y_{m}}{\partial x_{n}} \end{bmatrix},即Jacobian矩阵。

 

  • 矩阵关于向量的导数

设M×N矩阵矩阵求导与Hessian矩阵_第7张图片和p维向量矩阵求导与Hessian矩阵_第8张图片

矩阵求导与Hessian矩阵_第9张图片,其中\frac{\partial Y}{\partial x_{i}}=\begin{bmatrix} \frac{\partial y_{11}}{\partial x_{i}} &\frac{\partial y_{12}}{\partial x_{i}} &... &\frac{\partial y_{1n}}{\partial x_{i}} \\ \frac{\partial y_{21}}{\partial x_{i}} &\frac{\partial y_{22}}{\partial x_{i}} &... &\frac{\partial y_{2n}}{\partial x_{i}} \\ ... &... &... &... \\ \frac{\partial y_{m1}}{\partial x_{i}} &\frac{\partial y_{m2}}{\partial x_{i}} &... &\frac{\partial y_{mn}}{\partial x_{i}} \end{bmatrix}

 

  • 标量关于矩阵的导数

设标量y和M×N矩阵矩阵求导与Hessian矩阵_第10张图片

\frac{dy}{dX}=\begin{bmatrix} \frac{\partial y}{\partial x_{11}} &\frac{\partial y}{\partial x_{12}} &... &\frac{\partial y}{\partial x_{1n}} \\ \frac{\partial y}{\partial x_{21}} &\frac{\partial y}{\partial x_{22}} &... &\frac{\partial y}{\partial x_{2n}} \\ ... &... &... &... \\ \frac{\partial y}{\partial x_{m1}} &\frac{\partial y}{\partial x_{m2}} &... &\frac{\partial y}{\partial x_{mn}} \end{bmatrix}

 

  • 向量关于矩阵的导数

设p维向量矩阵求导与Hessian矩阵_第11张图片和M×N矩阵矩阵求导与Hessian矩阵_第12张图片

\frac{dy}{dX}=\begin{bmatrix} \frac{\partial y}{\partial x_{11}} &\frac{\partial y}{\partial x_{12}} &... &\frac{\partial y}{\partial x_{1n}} \\ \frac{\partial y}{\partial x_{21}} &\frac{\partial y}{\partial x_{22}} &... &\frac{\partial y}{\partial x_{2n}} \\ ... &... &... &... \\ \frac{\partial y}{\partial x_{m1}} &\frac{\partial y}{\partial x_{m2}} &... &\frac{\partial y}{\partial x_{mn}} \end{bmatrix},其中矩阵求导与Hessian矩阵_第13张图片

 

  • 矩阵关于矩阵的导数

设P×Q矩阵矩阵求导与Hessian矩阵_第14张图片和M×N矩阵矩阵求导与Hessian矩阵_第15张图片

\frac{dY}{dX}=(\frac{\partial Y}{\partial x_{ij}})_{pm\times qn}=\begin{bmatrix} \frac{\partial Y}{\partial x_{11}} &\frac{\partial Y}{\partial x_{12}} &... &\frac{\partial Y}{\partial x_{1n}} \\ \frac{\partial Y}{\partial x_{21}} &\frac{\partial Y}{\partial x_{22}} &... &\frac{\partial Y}{\partial x_{2n}} \\ ... &... &... &... \\ \frac{\partial Y}{\partial x_{m1}} &\frac{\partial Y}{\partial x_{m2}} &... &\frac{\partial Y}{\partial x_{mn}} \end{bmatrix}

其中\frac{\partial Y}{\partial x_{ij}}=\begin{bmatrix} \frac{\partial y_{11}}{\partial x_{ij}} &\frac{\partial y_{12}}{\partial x_{ij}} &... &\frac{\partial y_{1q}}{\partial x_{ij}} \\ \frac{\partial y_{21}}{\partial x_{ij}} &\frac{\partial y_{22}}{\partial x_{ij}} &... &\frac{\partial y_{2q}}{\partial x_{ij}} \\ ... &... &... &... \\ \frac{\partial y_{p1}}{\partial x_{ij}} &\frac{\partial y_{p2}}{\partial x_{ij}} &... &\frac{\partial y_{pq}}{\partial x_{ij}} \end{bmatrix}

 

  • Hessian矩阵

矩阵求导与Hessian矩阵_第16张图片

The above Hessian is of the the function f:\mathbb{R}''\rightarrow \mathbb{R} where all second order partial derivatives of f exist and are continuous throughout it's domain & the function is f(x_{1},x_{2},...,x_{n})

https://brilliant.org/wiki/hessian-matrix/

 

你可能感兴趣的:(数学基础)