【上】CS229 吴恩达机器学习 习题作业答案 problem sets 03 PS03（全部问题解答，欢迎各位前辈指教）

1. A Simple Neural Network

(a)

首先写出forward过程：
$$\begin{array}{rl}{z}^{[1]}& =W^{[1]}x+W_0^{[1]}\\ h& =\sigma \left(z^{[1]}\right)\\ z^{[2]}& =W^{[2]}h+W_0^{[2]}\\ o& =\sigma \left(z^{[2]}\right)\end{array}$$
损失函数为：
$$\begin{array}{rl}\ell & =\frac{1}{m}\sum _{i=1}^m{\left(o^{(i)}-y^{(i)}\right)}^2=\frac{1}{m}\sum _{i=1}^m{J}^{(i)}\end{array}$$
对于一个样本，通过链式法则，先求对于$W^{[2]}$的导数，利用其中结果，再求对于$W^{[1]}$的导数：
$$\begin{gathered} \frac{\partial J}{\partial W^{[1]}}=\frac{\partial J}{\partial o}\frac{\partial o}{\partial z^{[2]}}\frac{\partial z^{[2]}}{\partial h}\frac{\partial h}{\partial z^{[1]}}\frac{\partial z^{[1]}}{\partial W^{[1]}} \\ =2(o-y)o(1-o)x^{T}h\cdot (1-h)\cdot W^{[2]} \end{gathered}$$
其中"$\cdot$"表示element-wise乘法。

则，对于$w_{1,2}^{[1]}$,就是RHS所表示的(2x3)矩阵中的第（1，2）个元素，同时将上标(i)加入，表示第i个样本：
$$\frac{\partial J}{\partial w_{1,2}^{[1]}}=2\left(o^{(i)}-y^{(i)}\right)\cdot o^{(i)}\left(1-o^{(i)}\right)\cdot w_2^{[2]}\cdot h_2^{(i)}\left(1-h_2^{(i)}\right)\cdot x_1^{(i)}$$
其中$h_2=\sigma (w_{1,2}^{[1]}{x}_1+w_{2,2}^{[1]}{x}_2+w_{0,2}^{[1]})$

从而对于$\ell$的求导为：
$$\frac{\partial \ell }{\partial w_{1,2}^{[1]}}=\frac{2}{m}\sum _{i=1}^m\left(o^{(i)}-y^{(i)}\right)\cdot o^{(i)}\left(1-o^{(i)}\right)\cdot w_2^{[2]}\cdot h_2^{(i)}\left(1-h_2^{(i)}\right)\cdot x_1^{(i)}$$
则$w_{1,2}^{[1]}$的更新规则为：
$$w_{1,2}^{[1]}:=w_{1,2}^{[1]}-\alpha \frac{2}{m}\sum _{i=1}^m\left(o^{(i)}-y^{(i)}\right)\cdot o^{(i)}\left(1-o^{(i)}\right)\cdot w_2^{[2]}\cdot h_2^{(i)}\left(1-h_2^{(i)}\right)\cdot x_1^{(i)}$$

(b)

是可能的。可以将三个神经元看作三个独立的线性分类器，每个分类器代表一个超平面，分别落在散点图中0-1两类的三个三角形边边界上。则每个数据经过其中一个神经元的线性weights时，得出正负表示在超平面的上下（左右），然后利用step函数，直接将其划分为两类。在输出层线形计算部分，判断三个神经元是否都判断样本点在各个超平面1（0）一侧，如果都在则使其值为正（负或0），则最后经历一个step函数，将各自归类。100%可以保证是因为每个边界都线性可分。一个例子如下：

def optimal_step_weights():
    """Return the optimal weights for the neural network with a step activation function.
    
    This function will not be graded if there are no optimal weights.
    See the PDF for instructions on what each weight represents.
    
    The hidden layer weights are notated by [1] on the problem set and 
    the output layer weights are notated by [2].

    This function should return a dict with elements for each weight, see example_weights above.

    """
    w = example_weights()

    # *** START CODE HERE ***
    # x1 = 0.5 超平面
    w['hidden_layer_0_1'] = 0.5
    w['hidden_layer_1_1'] = -1
    w['hidden_layer_2_1'] = 0
    # x2 = 0.5 超平面
    w['hidden_layer_0_2'] = 0.5
    w['hidden_layer_1_2'] = 0
    w['hidden_layer_2_2'] = -1
    # x1 + x2 = 4 超平面
    w['hidden_layer_0_3'] = -4
    w['hidden_layer_1_3'] = 1
    w['hidden_layer_2_3'] = 1
    # 使得以上三个条件均为0的为0类
    w['output_layer_0'] = -0.5
    w['output_layer_1'] = 1
    w['output_layer_2'] = 1
    w['output_layer_3'] = 1
    # *** END CODE HERE ***

    return w

def example_weights():
    """This is an example function that returns weights.
    Use this function as a template for optimal_step_weights and optimal_sigmoid_weights.
    You do not need to modify this class for this assignment.
    """
    w = {}

    w['hidden_layer_0_1'] = 0
    w['hidden_layer_1_1'] = 0
    w['hidden_layer_2_1'] = 0
    w['hidden_layer_0_2'] = 0
    w['hidden_layer_1_2'] = 0
    w['hidden_layer_2_2'] = 0
    w['hidden_layer_0_3'] = 0
    w['hidden_layer_1_3'] = 0
    w['hidden_layer_2_3'] = 0

    w['output_layer_0'] = 0
    w['output_layer_1'] = 0
    w['output_layer_2'] = 0
    w['output_layer_3'] = 0

    return w

example_w = optimal_step_weights()
example_w

{'hidden_layer_0_1': 0.5,
 'hidden_layer_1_1': -1,
 'hidden_layer_2_1': 0,
 'hidden_layer_0_2': 0.5,
 'hidden_layer_1_2': 0,
 'hidden_layer_2_2': -1,
 'hidden_layer_0_3': -4,
 'hidden_layer_1_3': 1,
 'hidden_layer_2_3': 1,
 'output_layer_0': -0.5,
 'output_layer_1': 1,
 'output_layer_2': 1,
 'output_layer_3': 1}

©

不可能。这个在讲义中讲到了。如果没有在每层加激活函数（activation function），那么会导致本质上所有层等效为一个线性的运算过程。在这个题目中，如果隐藏层激活函数为线性，就是自身函数，则：
$$\begin{array}{rl}o& =\sigma \left(z^{[2]}\right)\\ & =\sigma \left(W^{[2]}h+W_0^{[2]}\right)\\ & =\sigma \left(W^{[2]}\left(W^{[1]}x+W_0^{[1]}\right)+W_0^{[2]}\right)\\ & =\sigma \left(W^{[2]}{W}^{[1]}x+W^{[2]}{W}_0^{[1]}+W_0^{[2]}\right)\\ & =\sigma \left({\tilde{W}}^{[1}+{\tilde{W}}_0\right)\end{array}$$
其中，$\tilde{W}=W^{[2]}{W}^{[1]}\text{ and }{\tilde{W}}_0=W^{[2]}{W}_0^{[1]}+W_0^{[2]}$，等效于只做一次线性分类，而图中可以看出，这是一个线性不可分数据集，因此无法完成100%正确分类的目标。

2. KL divergence and Maximum Likelihood

(a)

关键是利用Jensens不等式：
$$\begin{array}{rl}{D}_{\mathrm{K}\mathrm{L}}(P\Vert Q)& =\sum _{x\in \mathcal{X}}P(x)\log \frac{P(x)}{Q(x)}\\ & =-\sum _{x\in \mathcal{X}}P(x)\log \frac{Q(x)}{P(x)}\\ & =E\left[-\log \frac{Q(x)}{P(x)}\right]\\ & \ge -\log E\left[\frac{Q(x)}{P(x)}\right]\\ & =-\log \left(\sum _{x\in \mathcal{X}}P(x)\frac{Q(x)}{P(x)}\right)\\ & =-\log \sum _{x\in \mathcal{X}}Q(x)\\ & =-\log 1\\ & =0\end{array}$$
对于等号的取得，当$P=Q$时，$D_{\mathrm{K}\mathrm{L}}(P\Vert Q)={\sum }_{x\in \mathcal{X}}P(x)\log 1=0$；

而若$D_{\mathrm{K}\mathrm{L}}(P\Vert Q)=0$，则由Jensens不等式取等号的条件，$\frac{Q(x)}{P(x)}=E\left[\frac{Q(x)}{P(x)}\right]={\sum }_{x\in \mathcal{X}}P(x)\frac{P(x)}{Q(x)}={\sum }_{x\in \mathcal{X}}Q(x)=1$,即：$P=Q$；

所以，当且仅当$P=Q$，$D_{\mathrm{K}\mathrm{L}}(P\Vert Q)=0$

(b)

证明：
$$\begin{array}{rl}{D}_{\mathrm{K}\mathrm{L}}(P(X,Y)\Vert Q(X,Y))& =\sum _x\sum _{y}P(x,y)\log \frac{P(x,y)}{Q(x,y)}\\ & =\sum _x\sum _{y}P(x)P(y\mid x)\log \frac{P(x)P(y\mid x)}{Q(x)Q(y\mid x)}\\ & =\sum _x\sum _{y}P(x)P(y\mid x)\left(\log \frac{P(x)}{Q(x)}+\log \frac{P(y\mid x)}{Q(y\mid x)}\right)\\ & =\sum _x\sum _{y}P(x)P(y\mid x)\log \frac{P(x)}{Q(x)}+\sum _x\sum _{y}P(x)P(y\mid x)\log \frac{P(y\mid x)}{Q(y\mid x)}\\ & =\sum _{x}P(x)\log \frac{P(x)}{Q(x)}\sum _{y}P(y\mid x)+\sum _{x}P(x)\sum _{y}P(y\mid x)\log \frac{P(y\mid x)}{Q(y\mid x)}\\ & =\sum _{x}P(x)\log \frac{P(x)}{Q(x)}+\sum _{x}P(x)\left(\sum _{y}P(y\mid x)\log \frac{P(y\mid x)}{Q(y\mid x)}\right)\\ & =D_{\mathrm{K}\mathrm{L}}(P(X)\Vert Q(X))+D_{\mathrm{K}\mathrm{L}}(P(Y\mid X)\Vert Q(Y\mid X))\end{array}$$

©

证明：
$$\begin{array}{rl}\arg \underset{\theta }{\min }{D}_{\mathrm{K}\mathrm{L}}\left(\hat{P}\Vert P_{\theta }\right)& =\arg \underset{\theta }{\min }\sum _{x\in \mathcal{X}}\hat{P}(x)\frac{\log \hat{P}(x)}{P_{\theta }(x)}\\ & =\arg \underset{\theta }{\min }\sum _{x\in \mathcal{X}}\hat{P}(x)\log \hat{P}(x)-\sum _{x\in \mathcal{X}}\hat{P}(x)\log P_{\theta }(x)\\ & =\arg \underset{\theta }{\max }\sum _{x\in \mathcal{X}}\hat{P}(x)\log P_{\theta }(x)\\ & =\arg \underset{\theta }{\max }\sum _{x\in \mathcal{X}}\left(\frac{1}{m}\sum _{i=1}^{m}1\left\{x^{(i)}=x\right\}\right)\log P_{\theta }(x)\\ & =\arg \underset{\theta }{\max }\sum _{i=1}^m\log P_{\theta }\left(x^{(i)}\right)\end{array}$$

3.KL divergence, Fisher Information, and the Natural Gradient

(a)

证明：
$$\begin{array}{rl}{\nabla }_{\theta }\log p(y;\theta )& =\frac{{\nabla }_{\theta }p(y;\theta )}{p(y;\theta )}\\ {\mathbb{E}}_{y\sim p(y;\theta )}\left[{{\nabla }_{{\theta }^{\prime }}\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\right]& ={\mathbb{E}}_{y\sim p(y;\theta )}\left[\frac{{\nabla }_{\theta }p(y;\theta )}{p(y;\theta )}\right]\\ & ={\int }_{-\infty }^{\infty }p(y;\theta )\frac{{\nabla }_{\theta }p(y;\theta )}{p(y;\theta )}dy\\ & ={\int }_{-\infty }^{\infty }{\nabla }_{\theta }p(y;\theta )dy\\ & ={\nabla }_{\theta }{\int }_{-\infty }^{\infty }p(y;\theta )dy\\ & ={\nabla }_{\theta }1\\ & =0\end{array}$$

(b)

证明：
$$\begin{array}{c}\mathrm{Cov}[X]=E\left[(X-E[X])(X-E[X]{)}^T\right]\\ =E\left[X{X}^T\right]\text{ when }E[X]=0\\ \mathcal{I}(\theta )={\mathrm{Cov}}_{y\sim p(y;\theta )}\left[{{\nabla }_{{\theta }^{\prime }}\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\right]\\ ={\mathbb{E}}_{y\sim p(y;\theta )}\left[{{\nabla }_{{\theta }^{\prime }}\log p\left(y;{\theta }^{\prime }\right){\nabla }_{{\theta }^{\prime }}\log p{\left(y;{\theta }^{\prime }\right)}^T|}_{{\theta }^{\prime }=\theta }\right]\end{array}$$
其中用到了(a)中，score function的均值为0的结论。

©

证明：
$$\begin{array}{rl}& \frac{\partial \log p(y;\theta )}{\partial {\theta }_i}=\frac{1}{p(y;\theta )}\frac{\partial p(y;\theta )}{\partial {\theta }_i}\\ \mathcal{I}(\theta {)}_{ij}& ={\mathbb{E}}_{y\sim p(y;\theta )}{\left[{{\nabla }_{{\theta }^{\prime }}\log p\left(y;{\theta }^{\prime }\right){\nabla }_{{\theta }^{\prime }}\log p{\left(y;{\theta }^{\prime }\right)}^T|}_{{\theta }^{\prime }=\theta }\right]}_{ij}\\ =& {\mathbb{E}}_{y\sim p(y;\theta )}\left[\frac{\partial \log p(y;\theta )}{\partial {\theta }_i}\frac{\partial \log p(y;\theta )}{\partial {\theta }_j}\right]\\ =& {\mathbb{E}}_{y\sim p(y;\theta )}\left[\frac{1}{(p(y;\theta ){)}^2}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}\right]\\ \frac{{\partial }^2\log p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}& =-\frac{1}{(p(y;\theta ){)}^2}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}+\frac{1}{p(y;\theta )}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}\\ {\mathbb{E}}_{y\sim p(y;\theta )}{\left[-{{\nabla }_{{\theta }^{\prime }}^2\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\right]}_{ij}& ={\mathbb{E}}_{y\sim p(y;\theta )}\left[\frac{1}{(p(y;\theta ){)}^2}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}-\frac{1}{p(y;\theta )}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}\right]\\ & ={\mathbb{E}}_{y\sim p(y;\theta )}\left[\frac{1}{(p(y;\theta ){)}^2}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}\right]-{\mathbb{E}}_{y\sim p(y;\theta )}\left[\frac{1}{p(y;\theta )}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}\right]\\ & ={\mathbb{E}}_{y\sim p(y;\theta )}\left[\frac{1}{(p(y;\theta ){)}^2}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}\right]-{\int }_{-\infty }^{\infty }p(y;\theta )\frac{1}{p(y;\theta )}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}dy\\ & ={\mathbb{E}}_{y\sim p(y;\theta )}\left[\frac{1}{(p(y;\theta ){)}^2}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}\right]-\frac{{\partial }^2}{\partial {\theta }_i\partial {\theta }_j}{\int }_{-\infty }^{\infty }p(y;\theta )dy\\ & ={\mathbb{E}}_{y\sim p(y;\theta )}\left[\frac{1}{(p(y;\theta ){)}^2}\frac{{\partial }^{2}p(y;\theta )}{\partial {\theta }_i\partial {\theta }_j}\right]\\ & =\mathcal{I}(\theta {)}_{ij}\\ 从而：{\mathbb{E}}_{y\sim p(y;\theta )}\left[-{{\nabla }_{{\theta }^{\prime }}^2\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\right]=\mathcal{I}(\theta )& \end{array}$$

(d)

证明：
$$\begin{array}{rl}\log p(y;\tilde{\theta })& \approx \log p(y;\theta )+{(\tilde{\theta }-\theta {)}^T{\nabla }_{{\theta }^{\prime }}\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }+\frac{1}{2}(\tilde{\theta }-\theta {)}^T\left({{\nabla }_{{\theta }^{\prime }}^2\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\right)(\tilde{\theta }-\theta )\\ & =\log p(y;\theta )+{d^T{\nabla }_{{\theta }^{\prime }}\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }+\frac{1}{2}{d}^T\left({{\nabla }_{{\theta }^{\prime }}^2\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\right)d\end{array}$$
$$\begin{array}{rl}{\mathbb{E}}_{y\sim p(y;\theta )}[\log p(y;\tilde{\theta })]& ={\mathbb{E}}_{y\sim p(y;\theta )}[\log p(y;\theta )]+\frac{1}{2}{d}^T{\mathbb{E}}_{y\sim p(y;\theta )}\left[{{\nabla }_{{\theta }^{\prime }}^2\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\right]d\\ =& {\mathbb{E}}_{y\sim p(y;\theta )}[\log p(y;\theta )]+\frac{1}{2}{d}^T\mathcal{I}(\theta )d\\ D_{\mathrm{K}\mathrm{L}}\left(p_{\theta }\Vert p_{\theta +d}\right)& =D_{\mathrm{K}\mathrm{L}}\left(p_{\theta }\Vert p_{\tilde{\theta }}\right)\\ & ={\mathbb{E}}_{y\sim p(y;\theta )}[\log p(y;\theta )]-{\mathbb{E}}_{y\sim p(y;\theta )}[\log p(y;\tilde{\theta })]\\ & \approx \frac{1}{2}{d}^T\mathcal{I}(\theta )d\end{array}$$

(e)

第一步，用泰勒展开近似目标函数和约束：
$$\begin{array}{rl}\ell (\theta +d)& \approx \ell (\theta )+{d^T{\nabla }_{{\theta }^{\prime }}\ell \left({\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\\ & =\log p(y;\theta )+{d^T{\nabla }_{{\theta }^{\prime }}\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\\ & =\log p(y;\theta )+d^T\frac{{{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }}{p(y;\theta )}\\ & D_{\mathrm{K}\mathrm{L}}\left(p_{\theta }\Vert p_{\theta +d}\right)\approx \frac{1}{2}{d}^T\mathcal{I}(\theta )d\end{array}$$
第二步，写出拉格朗日函数：
$$\begin{array}{rl}\mathcal{L}(d,\lambda )& =\ell (\theta +d)-\lambda \left[D_{\mathrm{K}\mathrm{L}}\left(p_{\theta }\Vert p_{\theta +d}\right)-c\right]\\ & \approx \log p(y;\theta )+d^T\frac{{{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }}{p(y;\theta )}-\lambda \left[\frac{1}{2}{d}^T\mathcal{I}(\theta )d-c\right]\end{array}$$
第三步，拉格朗日函数对参数求导为0，其中关于d的求导为：
$$\begin{array}{rl}{\nabla }_d\mathcal{L}(d,\lambda )& \approx \frac{{{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }}{p(y;\theta )}-\lambda \mathcal{I}(\theta )d=0\\ \tilde{d}=\frac{1}{\lambda }\mathcal{I}(\theta {)}^{-1}\frac{{{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }}{p(y;\theta )}& \end{array}$$
则此时尽管不知$\lambda$的具体值，但是其为一个正实数，所以仍然已经得到了natural gradient的方向，下面利用关于$\lambda$求梯度的方程，
解出$\lambda$的具体值：
$$\begin{array}{rl}{\nabla }_{\lambda }\mathcal{L}(d,\lambda )& \approx c-\frac{1}{2}{d}^T\mathcal{I}(\theta )d\\ & =c-\frac{1}{2}\cdot \frac{1}{\lambda }\frac{{{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }^T}{p(y;\theta )}\mathcal{I}(\theta {)}^{-1}\cdot \mathcal{I}(\theta )\cdot \frac{1}{\lambda }\mathcal{I}(\theta {)}^{-1}\frac{{{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }}{p(y;\theta )}\\ & =c-{{\frac{1}{2{\lambda }^2(p(y;\theta ){)}^2}{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }^T\mathcal{I}(\theta {)}^{-1}{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\\ & =0\\ & \lambda =\sqrt{{{\frac{1}{2c(p(y;\theta ){)}^2}{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }^T\mathcal{I}(\theta {)}^{-1}{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }}\end{array}$$
从而可知natural gradient $d^{\ast }$:
$$\begin{array}{rl}{d}^{\ast }& =\sqrt{\frac{2c(p(y;\theta ){)}^2}{{{{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }^T\mathcal{I}(\theta {)}^{-1}{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }}}\mathcal{I}(\theta {)}^{-1}\frac{{{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }}{p(y;\theta )}\\ & ={\sqrt{\frac{2c}{{{{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }^T\mathcal{I}(\theta {)}^{-1}{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }}}\mathcal{I}(\theta {)}^{-1}{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\end{array}$$

(f)

由上一问：
$$\tilde{d}=\frac{1}{\lambda }\mathcal{I}(\theta {)}^{-1}\frac{{{\nabla }_{{\theta }^{\prime }}p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }}{p(y;\theta )}=\frac{1}{\lambda }\mathcal{I}(\theta {)}^{-1}{\nabla }_{{\theta }^{\prime }}logp\left(y;{\theta }^{\prime }\right){|}_{{\theta }^{\prime }=\theta }=\frac{1}{\lambda }\mathcal{I}(\theta {)}^{-1}{\nabla }_{{\theta }^{\prime }}log\ell ({\theta }^{\prime }){|}_{{\theta }^{\prime }=\theta }$$
又注意到：
$$\begin{array}{rl}\mathcal{I}(\theta )& ={\mathbb{E}}_{y\sim p(y;\theta )}\left[-{{\nabla }_{{\theta }^{\prime }}^2\log p\left(y;{\theta }^{\prime }\right)|}_{{\theta }^{\prime }=\theta }\right]\\ & ={\mathbb{E}}_{y\sim p(y;\theta )}\left[-{\nabla }_{\theta }^2\ell (\theta )\right]\\ & =-{\mathbb{E}}_{y\sim p(y;\theta )}[H]\end{array}$$
从而：
$$\begin{array}{rl}\theta :& =\theta +\tilde{d}\\ & =\theta +\frac{1}{\lambda }\mathcal{I}(\theta {)}^{-1}{\nabla }_{\theta }\ell (\theta )\\ & =\theta -\frac{1}{\lambda }{\mathbb{E}}_{y\sim p(y;\theta )}[H{]}^{-1}{\nabla }_{\theta }\ell (\theta )\end{array}$$
对于牛顿法，更新规则为：
$$\theta :=\theta -H^{-1}{\nabla }_{\theta }\ell (\theta )$$
可以发现更新的放向都是黑塞矩阵你矩阵与目标函数关于参数梯度的乘积方向。