最优控制 2：使用变分法求解最优控制问题

引言

上一篇博客粗略地讲了最优控制中不同情形下泛函取得极值的必要条件。但是那个所谓的“泛函”是比较抽象的，这个博客将会细化这个问题，并且将问题具象化为：如何使用变分法求解最优控制问题。即：最小化
$$J=\phi \left[x(t_1),t_1\right]+{\int }_{t_0}^{t_1}L(x,u,t)dt$$
其中，$\phi (\cdot )$是末端状态惩罚项，$u$是控制输入，并且系统要时时刻刻满足微分方程约束 $\dot{x}=f(x,t)$。

与上一个博客类似，这里还是分几种情况分别叙述。注意：所有的最优控制问题的初始时刻和初始状态都是已知的，这是合理且必须的 (啥时候开始、在哪开始都不知道，还玩个啥…)。

1. 末段时刻固定的最优问题解

这类问题的数学描述为：
$$\begin{array}{r}\begin{array}{rl}{u}^{\ast }& =\underset{u}{\mathrm{argmin}}\phi \left[x(t_1)\right]+{\int }_{t_{)}}^{t_1}L\left(x,u,t\right)dt\\ & s.t.\dot{x}=f(x,u,t),x_0=x(t_0),\psi \left[x(t_1)\right]=0\end{array}\end{array}$$
这实际上是一个带有等式约束的泛函极值的问题。他可以通过引入拉格朗日乘子 $\gamma$ 和 $\lambda (t)$，构造广义泛函 $J_a$，并定义哈密尔顿函数的方法来解决。

构造广义泛函如下：
$$\begin{array}{r}\begin{array}{rl}{J}_a& =\phi \left(x_1\right)+\gamma \psi (x_1)+{\int }_{t_{)}}^{t_1}\left\{L\left(x,u,t\right)+\lambda \left[f(x,u,t-\dot{x})\right]\right\}dt\end{array}\end{array}$$
定义哈密尔顿函数 (Hamiltonian function) 如下：
$$\begin{array}{r}\begin{array}{r}H(x,u,\lambda ,t)=L(x,u,t)+\lambda f(x,u,t)\end{array}\end{array}$$
将哈密尔顿函数代入 (2)，并进行一次分部积分有：
$$\begin{array}{r}\begin{array}{rl}{J}_a& =\phi \left(x_1\right)+\gamma \psi (x_1)+{\int }_{t_0}^{t_1}H(x,u,\gamma ,\lambda )-\lambda \dot{x}dt\\ & =\phi \left(x_1\right)+\gamma \psi (x_1)-{\lambda x|}_{t_0}^{t_1}+{\int }_{t_0}^{t_1}H(x,u,\gamma ,\lambda )+\dot{\lambda }xdt\end{array}\end{array}$$
这里需要计算 $J_a$ 的变分，注意 $J_a$ 仅仅会受 $\delta x$ 和 $\delta u$ 影响，而不会受 $\lambda$ 和 $\gamma$ 影响。这里推导一次，剩下的都类似~~
$$\begin{array}{r}\begin{array}{rl}{J}_a(x+\delta x,u+\delta u)& =\phi (x_1+\delta x_1)+\gamma \psi (x_1+\delta x_1)-{\lambda (x+\delta x)|}_{t_0}^{t_1}\\ & +{\int }_{t_0}^{t_1}H(x+\delta x,u+\delta u,\gamma ,\lambda )+\dot{\lambda }(x+\delta x)dt\end{array}\end{array}$$
太长了，写不下，(5) 中第一行记为 $J_{a_1}(x+\delta x,u+\delta u)=\widetilde{J_{a_1}}$，第二行记为 $J_{a_2}(x+\delta x,u+\delta u)=\widetilde{J_{a_2}}$。则有
J a 1 ~ = φ ( x 1 + δ x 1 ) + γ ψ ( x 1 + δ x 1 ) − λ x ∣ t 0 t 1 − λ ( t 1 ) δ x 1 = φ ( x 1 ) + ∂ φ ∂ x ∣ x = x 1 δ x 1 + γ ψ ( x 1 ) + γ ∂ ψ ∂ x ∣ x = x 1 δ x 1 \begin{align} \begin{aligned} \tilde{J_{a_1}} &= \varphi(x_1+\delta x_1)+\gamma\psi(x_1+\delta x_1)-\left.\lambda x\right|_{t_0}^{t_1}-\lambda(t_1)\delta x_1\\ &=\varphi(x_1)+\left.\frac{\partial\varphi}{\partial x}\right|_{x=x_1}\delta x_1+\gamma\psi(x_1)+\left.\gamma\frac{\partial\psi}{\partial x}\right|_{x=x_1}\delta x_1 \end{aligned} \end{align} Ja1​​~​​=φ(x1​+δx1​)+γψ(x1​+δx1​)−λx∣t0​t1​​−λ(t1​)δx1​=φ(x1​)+∂x∂φ​ ​x=x1​​δx1​+γψ(x1​)+γ∂x∂ψ​ ​x=x1​​δx1​​​​
$$\begin{array}{r}\begin{array}{rl}\widetilde{J_{a_2}}& ={\int }_{t_0}^{t_1}H(x+\delta x,u+\delta u,\gamma ,\lambda )+\dot{\lambda }(x+\delta x)dt\\ & ={\int }_{t_0}^{t_1}H(x,u,\gamma ,\lambda )+\dot{\lambda }xdt+{\int }_{t_0}^{t_1}\frac{\partial H}{\partial x}\delta x+\frac{\partial H}{\partial u}\delta u+\dot{\lambda }\delta xdt\end{array}\end{array}$$
则 $J_a$ 的变分为：
δ J a = J a 1 ~ + J a 2 ~ − J a = [ ∂ φ ∂ x + γ ∂ ψ ∂ x − λ ( t 1 ) ∣ t = t 1 , x = x 1 ] δ x 1 + ∫ t 0 t 1 [ ∂ H ∂ x + λ ˙ ] δ x + ∂ H ∂ u δ u d t \begin{align} \begin{aligned} \delta J_a&=\tilde{J_{a_1}}+\tilde{J_{a_2}}-J_a\\ &=\left[\left.\frac{\partial\varphi}{\partial x} + \gamma\frac{\partial\psi}{\partial x} - \lambda(t_1)\right|_{t=t_1,x=x_1}\right]\delta x_1\\ &+\int_{t_0}^{t_1}{\left[\frac{\partial H}{\partial x}+\dot{\lambda}\right]\delta x+\frac{\partial H}{\partial u}\delta u}dt \end{aligned} \end{align} δJa​​=Ja1​​~​+Ja2​​~​−Ja​=[∂x∂φ​+γ∂x∂ψ​−λ(t1​) ​t=t1​,x=x1​​]δx1​+∫t0​t1​​[∂x∂H​+λ˙]δx+∂u∂H​δudt​​​
很自然地，(8) 若要恒等于零，那么必须有以下必要条件成立
$$\begin{array}{r}\begin{array}{r}\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial x}+\dot{{\lambda }^{\ast }}(t)=0\end{array}\end{array}$$
$$\begin{array}{r}\begin{array}{r}\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial u}=0\end{array}\end{array}$$
∂ φ [ x ∗ ( t ) ] ∂ x + γ ∗ ∂ ψ [ x ∗ ( t ) ] ∂ x − λ ∗ ( t ) ∣ t = t 1 = 0 \begin{align} \begin{aligned} \left.\frac{\partial\varphi\left[x^*(t)\right]}{\partial x} + \gamma^*\frac{\partial\psi\left[x^*(t)\right]}{\partial x} - \lambda^*(t)\right|_{t=t_1}=0 \end{aligned} \end{align} ∂x∂φ[x∗(t)]​+γ∗∂x∂ψ[x∗(t)]​−λ∗(t) ​t=t1​​=0​​​
同时，根据 (3) 中 哈密尔顿函数的定义，有
$$\begin{array}{r}\begin{array}{r}{\dot{x}}^{\ast }=f\left(x^{\ast },u^{\ast },t\right)=\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial \lambda }\end{array}\end{array}$$

1.1 末端时刻固定，末端状态固定的最优控制的必要条件

1. 正则方程：
   $$\begin{array}{r}\begin{array}{rl}& \dot{{\lambda }^{\ast }}(t)=-\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial x}\\ & \dot{x^{\ast }}(t)=\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial \lambda }\end{array}\end{array}$$
2. 控制方程
   $$\begin{array}{r}\begin{array}{r}\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial u}=0\end{array}\end{array}$$
3. 边界条件 ($\psi $函数没了，$\delta x_1$ 也没了)
   $$\begin{array}{r}\begin{array}{r}{x}^{\ast }(t_0)=x_0,x^{\ast }(t_1)=x_1\end{array}\end{array}$$

1.2 末段时刻固定，终端状态自由的最优控制的必要条件

1. 正则方程：
   $$\begin{array}{r}\begin{array}{rl}& \dot{{\lambda }^{\ast }}(t)=-\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial x}\\ & \dot{x^{\ast }}(t)=\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial \lambda }\end{array}\end{array}$$
2. 控制方程
   $$\begin{array}{r}\begin{array}{r}\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial u}=0\end{array}\end{array}$$
3. 横截条件 (因为终端状态自由，所以 $\psi$ 那个就没了)
   $$\begin{array}{r}\begin{array}{rl}& \frac{\partial \phi \left[x^{\ast }(t_1)\right]}{\partial x}={\lambda }^{\ast }(t_1)\\ & x^{\ast }(t_0)=x_0\end{array}\end{array}$$

1.2 末段时刻固定，终端状态受约束的最优控制的必要条件

1. 正则方程：
   $$\begin{array}{r}\begin{array}{rl}& \dot{{\lambda }^{\ast }}(t)=-\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial x}\\ & \dot{x^{\ast }}(t)=\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial \lambda }\end{array}\end{array}$$
2. 控制方程
   $$\begin{array}{r}\begin{array}{r}\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial u}=0\end{array}\end{array}$$
3. 横截条件 (因为终端状态受 $\psi$ 函数约束，所以 $\psi$ 又回来了)
   $$\begin{array}{r}\begin{array}{rl}& \frac{\partial \phi \left[x^{\ast }(t_1)\right]}{\partial x}+\frac{\partial \psi \left[x^{\ast }(t_1)\right]}{\partial x}{\gamma }^{\ast }={\lambda }^{\ast }(t_1)\\ & x^{\ast }(t_0)=x_0,\psi \left[x^{\ast }(t_1)\right]=0\end{array}\end{array}$$

2. 末段时刻自由的最优控制问题解

算了，还是写一遍吧，万一以后我忘了，还能回来查。终端时刻自由比终端时刻固定多了一个终端时刻的变分 $\delta t_1$。
$$\begin{array}{r}\begin{array}{rl}{J}_a(x+\delta x,u+\delta u,t_1+\delta t_1)& =\phi (x_1+\delta x_1,t_1+\delta t_1)+\gamma \psi (x_1+\delta x_1,t_1+\delta t_1)\\ & +{\int }_{t_0}^{t_1+\delta t_1}H(x+\delta x,u+\delta u,\gamma ,\lambda )-\lambda (\dot{x}+\delta \dot{x})dt\end{array}\end{array}$$
同理，令 (19) 中第一行记为 $\widetilde{J_{a_1}}$，第二行记为 $\widetilde{J_{a_2}}$，则有
δ J a 1 ~ = φ ( x 1 + δ x 1 , t 1 + δ t 1 ) + γ ψ ( x 1 + δ x 1 , t 1 + δ t 1 ) = ∂ φ ∂ x ∣ x = x 1 δ x 1 + ∂ φ ∂ t 1 δ t 1 + γ ∂ ψ ∂ x ∣ x = x 1 δ x 1 + γ ∂ ψ ∂ t 1 δ t 1 \begin{align} \begin{aligned} \delta\tilde{J_{a_1}} &= \varphi(x_1+\delta x_1,t_1+\delta t_1)+\gamma\psi(x_1+\delta x_1,t_1+\delta t_1) \\ &= \left.\frac{\partial\varphi}{\partial x}\right|_{x=x_1}\delta x_1+\frac{\partial\varphi}{\partial t_1}\delta t_1+\gamma\left.\frac{\partial\psi}{\partial x}\right|_{x=x_1}\delta x_1+\gamma\frac{\partial\psi}{\partial t_1}\delta t_1 \end{aligned} \end{align} δJa1​​~​​=φ(x1​+δx1​,t1​+δt1​)+γψ(x1​+δx1​,t1​+δt1​)=∂x∂φ​ ​x=x1​​δx1​+∂t1​∂φ​δt1​+γ∂x∂ψ​ ​x=x1​​δx1​+γ∂t1​∂ψ​δt1​​​​
类似地，
$$\begin{array}{r}\begin{array}{rl}\delta \widetilde{J_{a_2}}& ={\int }_{t_0}^{t_1+\delta t_1}H(x+\delta x,u+\delta u,\gamma ,\lambda )-\lambda (\dot{x}+\delta \dot{x})dt\\ & -{\int }_{t_0}^{t_1}H(x,u,\gamma ,\lambda )+\lambda \dot{x}dt\\ & ={\int }_{t_0}^{t_1}H(x+\delta x,u+\delta u,\gamma ,\lambda )-\lambda (\dot{x}+\delta \dot{x})-H(x,u,\gamma ,\lambda )-\lambda \dot{x}dt\\ & +{\int }_{t_1}^{t_1+\delta t_1}H(x+\delta x,u+\delta u,\gamma ,\lambda )-\lambda (\dot{x}+\delta \dot{x})dt\\ & ={\int }_{t_0}^{t_1}\frac{\partial H}{\partial x}\delta x+\frac{\partial H}{\partial u}\delta u-\lambda \delta \dot{x}dt+\left[H(x_1+\theta \delta x_1,u+\theta \delta u,\gamma ,\lambda )-\lambda {\dot{x}}_1\right]\delta t_1\\ & =-\lambda \delta x(t_1)+{\int }_{t_0}^{t_1}\left(\frac{\partial H}{\partial x}+\dot{\lambda }\right)\delta x+\frac{\partial H}{\partial u}\delta udt+\left[H(x_1,u,\gamma ,\lambda )-\lambda {\dot{x}}_1\right]\delta t_1\end{array}\end{array}$$
这里需要复习一下，上篇博客第二个图对应的近似公式：
$$\delta x(t_1)=\delta x_1-\dot{x}(t_1)\cdot \delta t_1$$
把它带入到 (21) 中，进而，
δ J a = δ J a 1 ~ + δ J a 2 ~ = ∂ φ ∂ x ∣ x = x 1 δ x 1 + ∂ φ ∂ t 1 δ t 1 + γ ∂ ψ ∂ x ∣ x = x 1 δ x 1 + γ ∂ ψ ∂ t 1 δ t 1 − λ δ x 1 + ∫ t 0 t 1 ( ∂ H ∂ x + λ ˙ ) δ x + ∂ H ∂ u δ u d t + [ H ( x 1 , u , γ , λ ) − λ x ˙ 1 ] δ t 1 = [ ∂ φ ( x 1 ) ∂ x 1 + γ ∂ ψ ( x 1 ) ∂ x 1 − λ ( t 1 ) ] δ x 1 + [ ∂ φ ( t 1 ) ∂ t 1 + γ ∂ ψ ( t 1 ) ∂ t 1 + H ( t 1 ) ] δ t 1 + ∫ t 0 t 1 ( ∂ H ∂ x + λ ˙ ) δ x + ∂ H ∂ u δ u d t \begin{align} \begin{aligned} \delta J_a &= \delta\tilde{J_{a_1}}+\delta\tilde{J_{a_2}}\\ &= \left.\frac{\partial\varphi}{\partial x}\right|_{x=x_1}\delta x_1+\frac{\partial\varphi}{\partial t_1}\delta t_1+\gamma\left.\frac{\partial\psi}{\partial x}\right|_{x=x_1}\delta x_1+\gamma\frac{\partial\psi}{\partial t_1}\delta t_1\\ &-\lambda\delta x_1+\int_{t_0}^{t_1}{\left(\frac{\partial H}{\partial x}+\dot{\lambda}\right)\delta x+\frac{\partial H}{\partial u}\delta u}dt+\left[H(x_1,u,\gamma,\lambda)-\lambda\dot{x}_1\right]\delta t_1\\ &=\left[\frac{\partial\varphi(x_1)}{\partial x_1}+\gamma\frac{\partial\psi(x_1)}{\partial x_1}-\lambda(t_1)\right]\delta x_1+\left[\frac{\partial\varphi(t_1)}{\partial t_1}+\gamma\frac{\partial\psi(t_1)}{\partial t_1}+H(t_1)\right]\delta t_1\\ &+\int_{t_0}^{t_1}{\left(\frac{\partial H}{\partial x}+\dot{\lambda}\right)\delta x+\frac{\partial H}{\partial u}\delta u}dt \end{aligned} \end{align} δJa​​=δJa1​​~​+δJa2​​~​=∂x∂φ​ ​x=x1​​δx1​+∂t1​∂φ​δt1​+γ∂x∂ψ​ ​x=x1​​δx1​+γ∂t1​∂ψ​δt1​−λδx1​+∫t0​t1​​(∂x∂H​+λ˙)δx+∂u∂H​δudt+[H(x1​,u,γ,λ)−λx˙1​]δt1​=[∂x1​∂φ(x1​)​+γ∂x1​∂ψ(x1​)​−λ(t1​)]δx1​+[∂t1​∂φ(t1​)​+γ∂t1​∂ψ(t1​)​+H(t1​)]δt1​+∫t0​t1​​(∂x∂H​+λ˙)δx+∂u∂H​δudt​​​
与之前同理，若要实现最优控制，那么两部分变分必须都恒为零才行。下边分别讨论

2.1 终端时刻自由，终端状态固定 ($\delta x_1$ 没了，但是只是固定，并不是受约束)

1. 正则方程
   $$\begin{array}{r}\begin{array}{rl}& \dot{{\lambda }^{\ast }}(t)=-\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial x}\\ & \dot{x^{\ast }}(t)=\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial \lambda }\end{array}\end{array}$$
2. 控制方程
   $$\begin{array}{r}\begin{array}{r}\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial u}=0\end{array}\end{array}$$
3. 边界条件
   $$\begin{array}{r}\begin{array}{r}{x}^{\ast }(t_0)=x_0,x^{\ast }(t_1)=x_1\end{array}\end{array}$$
4. 哈密尔顿函数终值条件 ($\psi$ 函数没了)
   $$\begin{array}{r}\begin{array}{r}H(t_1)=-\frac{\partial \phi }{\partial t_1}\end{array}\end{array}$$

2.2 终端时刻自由，终端状态自由 ($\delta x_1$ 又回来了，但是 $\psi$ 还是没有)

1. 正则方程
   $$\begin{array}{r}\begin{array}{rl}& \dot{{\lambda }^{\ast }}(t)=-\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial x}\\ & \dot{x^{\ast }}(t)=\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial \lambda }\end{array}\end{array}$$
2. 控制方程
   $$\begin{array}{r}\begin{array}{r}\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial u}=0\end{array}\end{array}$$
3. 横截条件
   $$\begin{array}{r}\begin{array}{r}{x}^{\ast }(t_0)=x_0,{\lambda }^{\ast }(t_1)=\frac{\partial \phi }{\partial x^{\ast }(t_1)}\end{array}\end{array}$$
4. 哈密尔顿函数终值条件 ($\psi$ 函数没了)
   $$\begin{array}{r}\begin{array}{r}H(t_1)=-\frac{\partial \phi }{\partial t_1}\end{array}\end{array}$$

2.3 终端时刻自由，终端状态受约束 (这回 $\psi$ 回来了)

1. 正则方程
   $$\begin{array}{r}\begin{array}{rl}& \dot{{\lambda }^{\ast }}(t)=-\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial x}\\ & \dot{x^{\ast }}(t)=\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial \lambda }\end{array}\end{array}$$
2. 控制方程
   $$\begin{array}{r}\begin{array}{r}\frac{\partial H\left(x^{\ast },u^{\ast },{\lambda }^{\ast },t\right)}{\partial u}=0\end{array}\end{array}$$
3. 横截条件
   $$\begin{array}{r}\begin{array}{r}{x}^{\ast }(t_0)=x_0,\psi \left[x^{\ast }(t_1),t_1\right]=0,\lambda (t_1)=\frac{\partial \phi }{\partial x^{\ast }(t_1)}+\gamma \frac{\partial \psi }{\partial x^{\ast }(t_1)}\end{array}\end{array}$$
4. 哈密尔顿函数终值条件 ($\psi$ 函数回来了)
   $$\begin{array}{r}\begin{array}{r}H(t_1)=-\frac{\partial \phi }{\partial t_1}-\gamma \frac{\partial \psi }{\partial t_1}\end{array}\end{array}$$

至此，本文分析了以下六种情况下的最优控制的必要条件：

1. 末端时刻固定，末端状态固定
2. 末端时刻固定，末端状态自由
3. 末端时刻固定，末端状态受约束
4. 末端时刻自由，末端状态固定
5. 末端时刻自由，末端状态自由
6. 末端时刻自由，末端状态受约束

结合上一篇博客，通过变分法求解最优控制必要条件的内容，基本结束。
哪里推导错的，欢迎批评指正~