Discrete Optimization

By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH

1. OptimizationÂ¶

An important tool in 1) engineering problem solving and 2) decision science

• peolple optimize
• nature optimizes

3 key components

1. objective
2. decision variable or unknown
3. constraints

Procedures

1. The process of identifying objective, variables, and constraints for a given problem is known as "modeling"
2. Once the model has been formulated, optimization algorithm can be used to find its solutions.

In mathematical expression

\begin{align} \min_{x} \quad &f(x) \\ \text{subject to} \quad &g_i(x) \leq 0, \qquad i=1,\cdots,m \end{align}

Remarks) equivalent

\begin{align} \min_{x} f(x) \quad&\leftrightarrow \quad \max_{x} -f(x)\\ \quad g_i(x) \leq 0\quad&\leftrightarrow \quad -g_i(x) \geq 0\\ h(x) = 0 \quad&\leftrightarrow \quad \begin{cases} h(x) \leq 0 \quad \text{and} \\ h(x) \geq 0 \end{cases} \end{align}

1.1. Modeling ExampleÂ¶

Manufacturer produces two products P and Q with machine A and B.

time of producing each unit of P

• in machine A: 50 min
• in machine B: 30 min

time of producing each unit of Q

• in machine A: 24 min
• in machine B: 33 min

working plan for a week $\begin{cases} \text{40 hrs of work on machine A} \\ \text{35 hrs of work on machine B} \end{cases}$

The week starts with $\begin{cases} \text{stock of 30 units of P} \\ \text{stock of 90 units of Q} \\ \text{demand of 75 units of P} \\ \text{demand of 95 units of Q} \\ \end{cases}$

Question: how to plan the production to end the week with the maximum stock?

Define decision variables

• $x$ = unit of P to be produced
• $y$ = unit of Q to be produced

\begin{align} \max_{x} \quad & (30+x-75)+(90+y-95) \\ \text{subject to} \quad & 50x+24y \leq 40 \times 60\\ & 30x + 33y \leq 35 \times 60\\ & x \geq 75-30\\ & y \geq 95-90 \end{align}

1.2. General or Standard FormsÂ¶

1) Linear programming \begin{align} \min_{x} \quad & f^Tx \\ \text{subject to} \quad & Ax \leq b\\ & A_{eq}x \leq b_{eq}\\ & LB \leq x \leq UB \end{align}

2) Integer programming \begin{align} \min_{x} \quad & f^Tx \\ \text{subject to} \quad & Ax \leq b\\ & A_{eq}x \leq b_{eq}\\ & LB \leq x \leq UB\\ & x \in \mathbb{Z} \text{(integer)} \end{align}

3) Mixed integer linear programming \begin{align} \min_{x,z} \quad & f^T \begin{bmatrix} x \\ z \end{bmatrix} \\ \text{subject to} \quad & A \begin{bmatrix} x \\ z \end{bmatrix} \leq b \\ & A_{eq} \begin{bmatrix} x \\ z \end{bmatrix} = b_{eq}\\ & LB \leq \begin{bmatrix} x \\ z \end{bmatrix} \leq UB \\ & x \in \mathbb{R} \text{(real)} \\ & z \in \mathbb{Z} \end{align}

4) Binary integer programming \begin{align} \min_{x} \quad & f^Tx \\ \text{subject to} \quad & Ax \leq b \\ & A_{eq}x = b_{eq} \\ & LB \leq x \leq UB \\ & x \in \{0,1\} \end{align}

5) Quadratic programming \begin{align} \min_{x} \quad & \frac{1}{2}x^THx + f^Tx \\ \text{subject to} \quad & Ax \leq b \\ & A_{eq}x = b_{eq} \\ & LB \leq x \leq UB \end{align}

2. Linear Programming (LP)Â¶

LP Example 1

\begin{align} \max_{x} \quad & x_1 + x_2 \\ \text{subject to} \quad & 2x_1 + x_2 \leq 29 \\ & x_1 + 2x_2 \leq 25 \\ & x_1 \geq 2 \\ & x_2 \geq 5 \end{align}

\begin{align} \implies \quad \min_{x} \quad & - \begin{bmatrix} 1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \\ \text{subject to} \quad & \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \leq \begin{bmatrix} 29 \\ 25 \end{bmatrix} \\ & \begin{bmatrix} 2 \\ 5 \end{bmatrix} \leq \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \leq \begin{bmatrix} \\ \\ \end{bmatrix} \end{align}
InÂ [1]:
f = -[1 1]';
A = [2 1;
1 2];
b = [29,25]';
lb = [2 5];

linprog(f,A,b,[],[],lb,[])

Out[1]:
Optimization terminated.

ans =

11.0000
7.0000

LP example 2

\begin{align} \min_{x} \quad & x_1 + x_2 + x_3 + x_4 + x_5 \\ \text{subject to} \quad & x_1 + x_2 = 100 \\ & x_3 + x_4 = 70 \\ & x_2 + x_3 + 2x_4 + 5x_5 = 250 \\ & x_{i} \geq 0 \quad (i = 1,2,\cdots,5) \end{align}

\begin{align} \implies \quad \min_{x} \quad & -\begin{bmatrix} 1 & 1 & 1 & 1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \end{bmatrix} \\ \text{subject to} \quad & \begin{bmatrix} 1 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 1 & 0 \\ 0 & 1 & 1 & 2 & 5 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \end{bmatrix} = \begin{bmatrix} 100 \\ 70 \\ 250 \end{bmatrix} \\ & \begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix} \leq \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \end{bmatrix} \leq \begin{bmatrix} \\ \\ \\ \\ \\ \end{bmatrix} \end{align}
InÂ [2]:
f = -[1 1 1 1 1]';
Aeq = [1 1 0 0 0;
0 0 1 1 0;
0 1 1 2 5];
beq = [100 70 250]';
lb = [0 0 0 0 0]';

linprog(f,[],[],Aeq,beq,lb,[])

Out[2]:
Optimization terminated.

ans =

100.0000
0.0000
70.0000
0.0000
36.0000

LP example 3

InÂ [3]:
f = -[1 1]';
A = [2 1];
b = log(0.25) - log(pi) + 2*log(2);
lb = [log(0.06) 0]';

x = linprog(f,A,b,[],[],lb,[])

d = exp(x(1))
n = exp(x(2))

Out[3]:
Optimization terminated.

x =

-2.8134
4.4821

d =

0.0600

n =

88.4194

3. Mixed Integer Linear Programming (MILP)Â¶

• Matlab command: intlinprog (available from MATLAB R2014b)
InÂ [4]:
help intlinprog

Out[4]:
INTLINPROG Mixed integer linear programming.

X = INTLINPROG(f,intcon,A,b) attempts to solve problems of the form

min f'*x    subject to:  A*x  <= b
x                       Aeq*x = beq
lb <= x <= ub
x(i) integer, where i is in the index
vector intcon (integer constraints)

X = INTLINPROG(f,intcon,A,b) solves the problem with integer variables
in the intcon vector and linear inequality constraints A*x <= b. intcon
is a vector of positive integers indicating components of the solution
X that must be integers. For example, if you want to constrain X(2) and
X(10) to be integers, set intcon to [2,10].

X = INTLINPROG(f,intcon,A,b,Aeq,beq) solves the problem above while
additionally satisfying the equality constraints Aeq*x = beq.

X = INTLINPROG(f,intcon,A,b,Aeq,beq,LB,UB) defines a set of lower and
upper bounds on the design variables, X, so that the solution is in the
range LB <= X <= UB. Use empty matrices for LB and UB if no bounds
exist. Set LB(i) = -Inf if X(i) is unbounded below; set UB(i) = Inf if
X(i) is unbounded above.

X = INTLINPROG(f,intcon,A,b,Aeq,beq,LB,UB,OPTIONS) minimizes with the
default optimization parameters replaced by values in OPTIONS, an
argument created with the OPTIMOPTIONS function. See OPTIMOPTIONS for
details.

X = INTLINPROG(PROBLEM) finds the minimum for PROBLEM. PROBLEM is a
structure with the vector 'f' in PROBLEM.f, the integer constraints in
PROBLEM.intcon, the linear inequality constraints in PROBLEM.Aineq and
PROBLEM.bineq, the linear equality constraints in PROBLEM.Aeq and
PROBLEM.beq, the lower bounds in PROBLEM.lb, the upper bounds in
PROBLEM.ub, the options structure in PROBLEM.options, and solver name
'intlinprog' in PROBLEM.solver.

[X,FVAL] = INTLINPROG(f,intcon,A,b,...) returns the value of the
objective function at X: FVAL = f'*X.

[X,FVAL,EXITFLAG] = INTLINPROG(f,intcon,A,b,...) returns an EXITFLAG
that describes the exit condition of INTLINPROG. Possible values of
EXITFLAG and the corresponding exit conditions are

2  Solver stopped prematurely. Integer feasible point found.
1  Optimal solution found.
0  Solver stopped prematurely. No integer feasible point found.
-2  No feasible point found.
-3  Root LP problem is unbounded.

[X,FVAL,EXITFLAG,OUTPUT] = INTLINPROG(f,A,b,...) returns a structure
OUTPUT containing information about the optimization process. OUTPUT
includes the number of integer feasible points found and the final gap
between internally calculated bounds on the solution. See the
documentation for a complete description.

Reference page in Help browser
doc intlinprog

MILP Example 1

\begin{align} \min_{x} \quad & \begin{bmatrix} -3 & -1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \\ \text{subject to} \quad & \begin{bmatrix} 17 & 11 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \leq 83 \\ & \begin{bmatrix} 0 \\ 0 \end{bmatrix} \leq \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \leq \begin{bmatrix} 3 \\ 7 \end{bmatrix} \\ & x_1, x_2 \in \mathbb{Z} \; \text{(integer)} \end{align}
InÂ [5]:
f = [-3 -1]';
intcon = 1:2;
A = [17 11];
b = 83;
LB = [0 0]';
UB = [3 7]';

x = intlinprog(f,intcon,A,b,[],[],LB,UB)

Out[5]:
LP:                Optimal objective value is -11.909091.

Cut Generation:    Applied 1 Gomory cut.
Lower bound is -11.000000.
Relative gap is 0.00%.

Optimal solution found.

Intlinprog stopped at the root node because the objective value is within a gap tolerance of the optimal value, options.TolGapAbs = 0
(the default value). The intcon variables are integer within tolerance, options.TolInteger = 1e-05 (the default value).

x =

3.0000
2.0000

MIP Example 2

\begin{align} \min_{x} \quad & \begin{bmatrix} -3 & -2 & -1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} \\ \text{subject to} \quad & \begin{bmatrix} 1 & 1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} \leq 7 \\ & \begin{bmatrix} 4 & 2 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = 12 \\ & 0 \leq \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} \\ & x_3 \in \{0,1\} \end{align}
InÂ [6]:
f = [-3; -2; -1];
intcon = 3;
A = [1, 1, 1];
b = 7;
Aeq = [4,2,1];
beq = 12;
lb = zeros(3,1);
ub = [Inf;Inf;1];

x = intlinprog(f,intcon,A,b,Aeq,beq,lb,ub)

Out[6]:
LP:                Optimal objective value is -12.000000.

Optimal solution found.

Intlinprog stopped at the root node because the objective value is within a gap tolerance of the optimal value, options.TolGapAbs = 0
(the default value). The intcon variables are integer within tolerance, options.TolInteger = 1e-05 (the default value).

x =

0
5.5000
1.0000

3.1. Knapsack problems revisitÂ¶

• There is a container(knapsack) of capacity C = 20. Further more, there is a set 6 of objects. Each object has a weight and a value.
• Determine the number of each item to include in a collection so that the total size is less than or equal to 20 and the total value is as large as possible.

items 1 2 3 4 5 6
weights 10 9 4 2 1 20
values 175 90 20 50 10 200

Question: Can we formulate this knapsack problem into a binary integer program?

$\Rightarrow$ decision variables $\quad x_{i} \in \{0,1\} \quad i = 1,\cdots,6$

\begin{align} \max_{x} \quad & 175x_1 + 90x_2 + 20x_3 + 50x_4 + 10x_5 + 200x_6 \\ \text{subject to} \quad & 10x_1 + 9x_2 + 4x_3 + 2x_4 + x_5 + 20x_6 \leq 20 \\ \end{align}

$\Rightarrow$ intlinprog in Matlab to solve

InÂ [7]:
n = 6;  % # of items
weights = [10 9 4 2 1 20];
values = [175 90 20 50 10 200];
maxWeight = 20;

% in a form of MILP

f = -values;        % max
A = weights;
b = maxWeight;
intcon = 1:n;
lb = zeros(n,1);
ub = ones(n,1);

x = intlinprog(f,intcon,A,b,[],[],lb,ub)

Out[7]:
LP:                Optimal objective value is -305.000000.

Cut Generation:    Applied 1 strong CG cut.
Lower bound is -275.000000.
Relative gap is 0.00%.

Optimal solution found.

Intlinprog stopped at the root node because the objective value is within a gap tolerance of the optimal value, options.TolGapAbs = 0
(the default value). The intcon variables are integer within tolerance, options.TolInteger = 1e-05 (the default value).

x =

1.0000
1.0000
0
0
1.0000
0

3.2. Factory and warehouseÂ¶

• A campany wants to $\begin{cases} \text{1. build new factory in Ulsan, Busan, or both} \\ \text{2. build new warehouse (only one has to be built)} \\ \text{3. warehouse must be built in a city of a new factory} \\ \end{cases}$
• Available capital is 10. We want to maximize total profit.

Profit Cost to build
1 Build a factory in Ulsan 7 5
2 Build a factory in Busan 5 3
3 Build a warehouse in Ulsan 6 5
4 Build a warehouse in Busan 4 2

• Form a binary integer programming to find the optimal locations of new factory and warehouse, and solve it using intlinprog in MATLAB.

$\quad\;$ (Hint: define binary decision variables, then use them to define objective function and constraints.)

InÂ [8]:
f = -[7 5 6 4]';
intcon = [1 2 3 4];
A = [5 3 5 2;
-1 -1 0 0;
-1 0 1 0;
0 -1 0 1];
b = [10;
-1;
0;
0];

Aeq = [0 0 1 1];
beq = [1];

lb = [0 0 0 0]';
ub = [1 1 1 1]';
[x fval] = intlinprog(f,intcon,A,b,Aeq,beq,lb,ub)

Out[8]:
LP:                Optimal objective value is -16.000000.

Optimal solution found.

Intlinprog stopped at the root node because the objective value is within a gap tolerance of the optimal value, options.TolGapAbs = 0
(the default value). The intcon variables are integer within tolerance, options.TolInteger = 1e-05 (the default value).

x =

1.0000
1.0000
0
1.0000

fval =

-16

3.3. Shortest pathÂ¶

• Binary integer programming can be applied to finding the shortest path in the graph.
• Define the binary decision variables and form the binary integer programming. Then you can solve it using intlinprog command in MATLAB

InÂ [9]:
f = [1 4 2 4 8 11 3]';
Aeq = [1 1 0 0 0 0 0
1 0 -1 0 0 -1 0
0 1 1 -1 -1 0 0
0 0 0 1 0 0 -1
0 0 0 0 1 1 1];
beq = [1 0 0 0 1]';

intcon = 1:7;
lb = zeros(7,1);
ub = ones(7,1);

x = intlinprog(f,intcon,[],[],Aeq,beq,lb,ub)

Out[9]:
LP:                Optimal objective value is 10.000000.

Optimal solution found.

Intlinprog stopped at the root node because the objective value is within a gap tolerance of the optimal value, options.TolGapAbs = 0
(the default value). The intcon variables are integer within tolerance, options.TolInteger = 1e-05 (the default value).

x =

1
0
1
1
0
0
1

3.4. Robot Path DesignÂ¶

• A circular mobile robot is to be navigated. The robot should take a path from point A to point B, avoiding the polygonal obstacles (hatched regions).
• Design a collision free-path the robot can take with the minimum length.