【最优化理论】4.2带约束最优化
带约束最优化
- 1.等式约束
- 2.不等式约束
- 3.优化的对偶理论
- 3.1原始问题
- 3.2对偶问题
1.等式约束
经典拉格朗日乘子法是下面的优化问题（注： x \boldsymbol{x} x是一个向量）
min x f ( x ) \min _{x} f(x) xminf(x)
s.t. g ( x ) = 0 \text {s.t.} \quad g(x)=0 s.t.g(x)=0直观上理解。最优解 x o p t i m a l \boldsymbol{x}_{optimal} xoptimal一定有这样的性质，以 x \boldsymbol{x} x是二维变量为例：（下图的约束曲线是 g ( x , y ) = c g(x,y)=c g(x,y)=c,当然也可以写成 g ( x , y ) − c = 0 g(x,y)−c=0 g(x,y)−c=0）
当等高线和曲线 g ( x , y ) = c g(x,y)=c g(x,y)=c相切，取得极值。两条曲线得切线方向是共线的，即法线方向也是共线的。得到下列式子：
{ ∇ f ( x ) = λ ∇ g ( x ) g ( x ) = 0 \left\{\begin{array}{l} \nabla f(\boldsymbol{x})=\lambda \nabla g(\boldsymbol{x}) \\ g(\boldsymbol{x})=0 \end{array}\right. {∇f(x)=λ∇g(x)g(x)=0
即得到拉格朗日函数：
L ( x , λ ) = f ( x ) + λ g ( x ) L(\boldsymbol{x}, \lambda)=f(\boldsymbol{x})+\lambda g(\boldsymbol{x}) L(x,λ)=f(x)+λg(x)
2.不等式约束
求下列非线性规划问题的K-T点；
min f ( x ) = 2 x 1 2 + 2 x 1 x 2 + x 2 − 10 x 1 − 10 x 2 s. t { x + x 2 ≤ 5 3 x + x 2 ≤ 6 \begin{aligned} &\min f(\boldsymbol{x})=2 x_{1}^{2}+2 x_{1} x_{2}+x_{2}-10 x_{1}-10 x_{2}\\ &\text { s. t} \left\{\begin{array}{l} x+x^{2} \leq 5 \\ 3 x+ x_{2} \leq 6 \end{array}\right. \end{aligned} minf(x)=2x12+2x1x2+x2−10x1−10x2 s. t{x+x2≤53x+x2≤6
解：将上述问题的约束条件改写为 g i ( x ) ≥ 0 g_i\left( \boldsymbol{x} \right) \ge 0 gi(x)≥0 的形式：
s . t { g 1 ( x ) = − x 1 2 − x 2 2 + 5 ≥ 0 g 2 ( x ) = − 3 x 1 2 − x 2 2 + 6 ≥ 0 s.t \begin{cases} g_1\left( \boldsymbol{x} \right) =-x_{1}^{2}-x_{2}^{2}+5\ge 0& \\ g_2\left( \boldsymbol{x} \right) =-3x_{1}^{2}-x_{2}^{2}+6\ge 0& \\ \end{cases} s.t{g1(x)=−x12−x22+5≥0g2(x)=−3x12−x22+6≥0设K-T点为 x ∗ = ( x 1 , x 2 ) T x^{*}=\left(x_{1}, x_{2}\right)^{T} x∗=(x1,x2)T有
∇ f ( x ∗ ) = [ 4 x 1 + 2 x 2 − 10 2 x 1 + 2 x 2 − 10 ] \nabla f\left(\boldsymbol{x^*} \right)=\left[\begin{array}{l} 4 x_{1}+2 x_{2}-10 \\ 2 x_{1}+2 x_{2}-10 \end{array}\right] ∇f(x∗)=[4x1+2x2−102x1+2x2−10]
∇ g 1 ( x ∗ ) = [ − 2 x 1 − 2 x 2 ] ∇ g 2 ( x ∗ ) = [ − 3 − 1 ] \begin{array}{c} \nabla g_{1}\left(x^{*}\right)=\left[\begin{array}{c} -2 x_{1} \\ -2 x_{2} \end{array}\right] \\ \nabla g_{2}\left(x^{*}\right)=\left[\begin{array}{c} -3 \\ -1 \end{array}\right] \end{array} ∇g1(x∗)=[−2x1−2x2]∇g2(x∗)=[−3−1]最后有:
{ 4 x 1 + 2 x 2 − 10 + 2 γ 1 x 1 + 3 γ 2 = 0 2 x 1 + 2 x 2 − 10 + 2 γ 1 x 2 + γ 2 = 0 γ 1 ( 5 − x 1 2 − x 2 2 ) = 0 γ 2 ( 6 − 3 x 1 − x 2 ) = 0 γ 1 ⩾ 0 γ 2 ⩾ 0 \left\{\begin{array}{l} 4 x_{1}+2 x_{2}-10+2 \gamma_{1} x_{1}+3 \gamma_{2}=0 \\ 2 x_{1}+2 x_{2}-10+2 \gamma_{1} x_{2}+\gamma_{2}=0 \\ \gamma_{1}\left(5-x_{1}^{2}-x_{2}^{2}\right)=0 \\ \gamma_{2}\left(6-3 x_{1}-x_{2}\right)=0 \\ \gamma_{1} \geqslant 0 \\ \gamma_{2} \geqslant 0 \end{array}\right. ⎩⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎧4x1+2x2−10+2γ1x1+3γ2=02x1+2x2−10+2γ1x2+γ2=0γ1(5−x12−x22)=0γ2(6−3x1−x2)=0γ1⩾0γ2⩾0
解集为：
{ x 1 = 1 x 2 = 2 γ 1 = 1 γ 2 = 0 \left\{\begin{array}{l} x_{1}=1 \\ x_{2}=2 \\ \gamma_{1}=1 \\ \gamma_{2}=0 \end{array}\right. ⎩⎪⎪⎨⎪⎪⎧x1=1x2=2γ1=1γ2=0
3.优化的对偶理论
3.1原始问题
假设 f ( x ) , c i ( x ) , h j ( x ) f(x),c_i(x),h_j(x) f(x),ci(x),hj(x)是定义 R n \boldsymbol R^n Rn上的连续可微函数。考虑约束最优化问题
min x ∈ R n f ( x ) \underset{x\in \mathbf{R}^n}{\min}f\left( x \right) x∈Rnminf(x)
s . t { c i ( x ) ≤ 0 , i = 1 , 2 , ⋯ , k h j ( x ) = 0 , j = 1 , 2 , ⋯ , l s.t\left\{ \begin{array}{l} c_i\left( x \right) \le 0,i=1,2,\cdots ,k\\ h_j\left( x \right) =0,j=1,2,\cdots ,l\\ \end{array} \right. s.t{ci(x)≤0,i=1,2,⋯,khj(x)=0,j=1,2,⋯,l称此约束最优化问题为原始最优化问题或原始问题.
首先，引进广义拉格朗日函数(generalized Lagrange function)
L ( x , α , β ) = f ( x ) + ∑ i = 1 k α i c i ( x ) + ∑ j = 1 l β j h j ( x ) L(x, \alpha, \beta)=f(x)+\sum_{i=1}^{k} \alpha_{i} c_{i}(x)+\sum_{j=1}^{l} \beta_{j} h_{j}(x) L(x,α,β)=f(x)+i=1∑kαici(x)+j=1∑lβjhj(x)
这里， x = ( x ( 1 ) , x ( 2 ) , ⋯ , x ( n ) ) T ∈ R n , α i , β j x=\left(x^{(1)}, x^{(2)}, \cdots, x^{(n)}\right)^{T} \in \mathbf{R}^{n}, \alpha_{i}, \beta_{j} x=(x(1),x(2),⋯,x(n))T∈Rn,αi,βj是拉格朗日乘子， α i ⩾ 0 \alpha_{i} \geqslant 0 αi⩾0.考虑 x x x的函数：
θ p ( x ) = max α ; β ; a i ≥ 0 L ( x , α , β ) \theta _p\left( x \right) =\underset{\alpha ;\beta ;a_i\ge 0}{\max}L\left( x,\alpha ,\beta \right) θp(x)=α;β;ai≥0maxL(x,α,β)
θ P ( x ) = { f ( x ) , x 满足居始问題约束 + ∞ , 其他 \theta_{P}(x)=\left\{\begin{array}{ll} f(x), & x \text { 满足居始问題约束 } \\ +\infty, & \text { 其他 } \end{array}\right. θP(x)={f(x),+∞,x 满足居始问題约束 其他 所以如果考虑极小化问题
min x θ p ( x ) = min x max α ; β ; a i ≥ 0 L ( x , α , β ) \underset{x}{\min}\theta _p\left( x \right) =\underset{x}{\min}\underset{\alpha ;\beta ;a_i\ge 0}{\max}L\left( x,\alpha ,\beta \right) xminθp(x)=xminα;β;ai≥0maxL(x,α,β)
3.2对偶问题
定义
θ D ( x ) = min α ; β ; a i ≥ 0 L ( x , α , β ) \theta _D\left( x \right) =\underset{\alpha ;\beta ;a_i\ge 0}{\min}L\left( x,\alpha ,\beta \right) θD(x)=α;β;ai≥0minL(x,α,β)再考虑极大化 θ D ( x ) = min α ; β ; a i ≥ 0 L ( x , α , β ) \theta _D\left( x \right) =\underset{\alpha ;\beta ;a_i\ge 0}{\min}L\left( x,\alpha ,\beta \right) θD(x)=α;β;ai≥0minL(x,α,β)即
max α ; β ; a i ≥ 0 θ p ( x ) = max α ; β ; a i ≥ 0 min x L ( x , α , β ) \underset{\alpha ;\beta ;a_i\ge 0}{\max}\theta _p\left( x \right) =\underset{\alpha ;\beta ;a_i\ge 0}{\max}\underset{x}{\min}L\left( x,\alpha ,\beta \right) α;β;ai≥0maxθp(x)=α;β;ai≥0maxxminL(x,α,β)
