拉格朗日松弛(lagrangian relaxation) - An Overview

拉格朗日松弛(lagrangian relaxation) - An Overview

---

1. Application

Relaxation is a commonly used method in integer or mixed integer programming. The author’s understanding of relaxation is that it relaxes requirements for some complex constraints or variables and preserve some of the original problem. In this way it can get a bound for the original problem.

Lagrangian Relaxation is applied to relax the complex contraints, which are added into the objective function as a penalty. And the relaxed problem becomes an easier problem with simple constraints that can be solved swiftly. Or it can be divided into simple sub-problems with polynomial time algorithm that can be applied.

So it should be known that the lagrangian relaxation obtains a lower bound of the original minimum problem, that is, an infeasible solution quite possibly.

Therefore, after getting the optimal solution of lagrangian relaxation problem, we still need a heuristic algorithm to repair the obtained infeasible solution into a feasible solution, which can help get a upper bound of the original problem.

2. Constraints relaxation

Assume that we have the following problem:

$$\begin{gathered} Z_{IP}=min{c}^{T}x+d^{T}y \\ Ax\ge b \\ Bx\ge d \\ x\in {\mathbb{Z}}^{+} \end{gathered}$$

It has 2 constraints, and we can apply lagrangian relaxation by add the constraint $Ax\ge b$ to the objective function.

And we get the lagrangian relaxation problem.
$$\begin{gathered} Z_{LR}(\lambda )=min{c}^{T}x+{\lambda }^T(b-Ax) \\ Bx\ge d \\ x\in {\mathbb{Z}}^{+} \end{gathered}$$

$\lambda$ is called the lagrange multiplier.

It should be noticed that the constraint decides whether $\lambda$ is positive or negative.

We can understand ${\lambda }^T(b-Ax)$ as the penalty function. Here $Ax\ge b$, and $b-Ax\le 0$. So $\lambda$ is positive. Then, ${\lambda }^T(b-Ax)$ is negative, the LR is the lower bound of the original problem.

Similarly, if the penalty function transformed from a constraint is positive, $\lambda$ should be negative. If the penalty function is zero, there is no demands on $\lambda$.

And obviously, here we have

$$Z_{LR}(\lambda )\le Z_{IP}$$

For every $\lambda$, there is a lagrangian relaxation problem $Z_{LR}(\lambda )$. So we need to find the best $\lambda$ to make the $Z_{LR}(\lambda )$ big enough, which leads us to the dual problem $Z_D=ma{x}_{\lambda \ge 0}{Z}_{LR}(\lambda )$.

Here subgradient algorithm is usually applied to solve the dual problem and update Lagrange multipliers.

3. Lagrangian Decomposition

For some problems with linked/coupling constraints, for example

$$\begin{gathered} Z_{IP}=min{c}^{T}x \\ Ax\ge b \\ x=y \\ By\ge d \\ x,y\in {\mathbb{Z}}^{+} \end{gathered}$$

Here constraint $x=y$ links $x$ and $y$.

So if we relax this constraint, we can get,

$$\begin{gathered} Z_{LR}(\lambda )=min{c}^{T}x+{\lambda }^T(x-y) \\ Ax\ge b \\ By\ge d \\ x,y\in {\mathbb{Z}}^{+} \end{gathered}$$

We can divide it into 2 sub-problems,

$$\begin{gathered} Z_{LR1}(\lambda )=min{c}^{T}x+{\lambda }^{T}x \\ Ax\ge b \\ x\in {\mathbb{Z}}^{+} \end{gathered}$$

$$\begin{gathered} Z_{LR2}(\lambda )=min-{\lambda }^{T}y \\ By\ge d \\ y\in {\mathbb{Z}}^{+} \end{gathered}$$

---

4. Conclusion

Lagrangian relaxation is a practical, simple and widely used method in problems, such as the (capacitated) facility location problem. However, as any of the method, there is alway huge gap between theory to practice.

For more detailed theory learning for beginners:
1. 整数规划的拉格朗日松弛（理论分析+Python代码实现） - 王源
2. 拉格朗日松弛-刘林冬

For code implementation learning, I strongly recommend that readers refer to the below articles:
1. 使用拉格朗日松弛求解约束最短路径问题 ( Matlab) - Mingrui Yu
2. Lagrange Relaxation求解TSP ( Java) - 数据魔术师