Discrete Optimization



By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH

Table of Contents

1. Optimization

An important tool in 1) engineering problem solving and 2) decision science

  • peolple optimize
  • nature optimizes

3 key components

  1. objective
  2. decision variable or unknown
  3. constraints

Procedures

  1. The process of identifying objective, variables, and constraints for a given problem is known as "modeling"
  2. Once the model has been formulated, optimization algorithm can be used to find its solutions.

In mathematical expression


$$\begin{align} \min_{x} \quad &f(x) \\ \text{subject to} \quad &g_i(x) \leq 0, \qquad i=1,\cdots,m \end{align} $$


Remarks) equivalent


$$\begin{align} \min_{x} f(x) \quad&\leftrightarrow \quad \max_{x} -f(x)\\ \quad g_i(x) \leq 0\quad&\leftrightarrow \quad -g_i(x) \geq 0\\ h(x) = 0 \quad&\leftrightarrow \quad \begin{cases} h(x) \leq 0 \quad \text{and} \\ h(x) \geq 0 \end{cases} \end{align} $$

1.1. Modeling Example

Manufacturer produces two products P and Q with machine A and B.

time of producing each unit of P

  • in machine A: 50 min
  • in machine B: 30 min

time of producing each unit of Q

  • in machine A: 24 min
  • in machine B: 33 min

working plan for a week $\begin{cases} \text{40 hrs of work on machine A} \\ \text{35 hrs of work on machine B} \end{cases}$

The week starts with $\begin{cases} \text{stock of 30 units of P} \\ \text{stock of 90 units of Q} \\ \text{demand of 75 units of P} \\ \text{demand of 95 units of Q} \\ \end{cases}$

Question: how to plan the production to end the week with the maximum stock?

Define decision variables

  • $x$ = unit of P to be produced
  • $y$ = unit of Q to be produced


$$\begin{align} \max_{x} \quad & (30+x-75)+(90+y-95) \\ \text{subject to} \quad & 50x+24y \leq 40 \times 60\\ & 30x + 33y \leq 35 \times 60\\ & x \geq 75-30\\ & y \geq 95-90 \end{align} $$

1.2. General or Standard Forms

1) Linear programming $$\begin{align} \min_{x} \quad & f^Tx \\ \text{subject to} \quad & Ax \leq b\\ & A_{eq}x \leq b_{eq}\\ & LB \leq x \leq UB \end{align} $$


2) Integer programming $$\begin{align} \min_{x} \quad & f^Tx \\ \text{subject to} \quad & Ax \leq b\\ & A_{eq}x \leq b_{eq}\\ & LB \leq x \leq UB\\ & x \in \mathbb{Z} \text{(integer)} \end{align} $$


3) Mixed integer linear programming $$\begin{align} \min_{x,z} \quad & f^T \begin{bmatrix} x \\ z \end{bmatrix} \\ \text{subject to} \quad & A \begin{bmatrix} x \\ z \end{bmatrix} \leq b \\ & A_{eq} \begin{bmatrix} x \\ z \end{bmatrix} = b_{eq}\\ & LB \leq \begin{bmatrix} x \\ z \end{bmatrix} \leq UB \\ & x \in \mathbb{R} \text{(real)} \\ & z \in \mathbb{Z} \end{align} $$


4) Binary integer programming $$\begin{align} \min_{x} \quad & f^Tx \\ \text{subject to} \quad & Ax \leq b \\ & A_{eq}x = b_{eq} \\ & LB \leq x \leq UB \\ & x \in \{0,1\} \end{align} $$


5) Quadratic programming $$\begin{align} \min_{x} \quad & \frac{1}{2}x^THx + f^Tx \\ \text{subject to} \quad & Ax \leq b \\ & A_{eq}x = b_{eq} \\ & LB \leq x \leq UB \end{align} $$

2. Linear Programming (LP)

LP Example 1


$$\begin{align} \max_{x} \quad & x_1 + x_2 \\ \text{subject to} \quad & 2x_1 + x_2 \leq 29 \\ & x_1 + 2x_2 \leq 25 \\ & x_1 \geq 2 \\ & x_2 \geq 5 \end{align} $$


$$\begin{align} \implies \quad \min_{x} \quad & - \begin{bmatrix} 1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \\ \text{subject to} \quad & \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \leq \begin{bmatrix} 29 \\ 25 \end{bmatrix} \\ & \begin{bmatrix} 2 \\ 5 \end{bmatrix} \leq \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \leq \begin{bmatrix} \\ \\ \end{bmatrix} \end{align} $$
In [1]:
f = -[1 1]';
A = [2 1;
    1 2];
b = [29,25]';
lb = [2 5];

linprog(f,A,b,[],[],lb,[])
Out[1]:
Optimization terminated.

ans =

   11.0000
    7.0000

LP example 2


$$\begin{align} \min_{x} \quad & x_1 + x_2 + x_3 + x_4 + x_5 \\ \text{subject to} \quad & x_1 + x_2 = 100 \\ & x_3 + x_4 = 70 \\ & x_2 + x_3 + 2x_4 + 5x_5 = 250 \\ & x_{i} \geq 0 \quad (i = 1,2,\cdots,5) \end{align} $$



$$\begin{align} \implies \quad \min_{x} \quad & -\begin{bmatrix} 1 & 1 & 1 & 1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \end{bmatrix} \\ \text{subject to} \quad & \begin{bmatrix} 1 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 1 & 0 \\ 0 & 1 & 1 & 2 & 5 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \end{bmatrix} = \begin{bmatrix} 100 \\ 70 \\ 250 \end{bmatrix} \\ & \begin{bmatrix} 0 \\ 0 \\ 0 \\ 0 \\ 0 \end{bmatrix} \leq \begin{bmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ x_5 \end{bmatrix} \leq \begin{bmatrix} \\ \\ \\ \\ \\ \end{bmatrix} \end{align} $$
In [2]:
f = -[1 1 1 1 1]';
Aeq = [1 1 0 0 0;
    0 0 1 1 0;
    0 1 1 2 5];
beq = [100 70 250]';
lb = [0 0 0 0 0]';

linprog(f,[],[],Aeq,beq,lb,[])
Out[2]:
Optimization terminated.

ans =

  100.0000
    0.0000
   70.0000
    0.0000
   36.0000

LP example 3

In [3]:
f = -[1 1]';
A = [2 1];
b = log(0.25) - log(pi) + 2*log(2);
lb = [log(0.06) 0]';

x = linprog(f,A,b,[],[],lb,[])

d = exp(x(1))
n = exp(x(2))
Out[3]:
Optimization terminated.

x =

   -2.8134
    4.4821


d =

    0.0600


n =

   88.4194

3. Mixed Integer Linear Programming (MILP)

  • Matlab command: intlinprog (available from MATLAB R2014b)
In [4]:
help intlinprog
Out[4]:
INTLINPROG Mixed integer linear programming.
 
    X = INTLINPROG(f,intcon,A,b) attempts to solve problems of the form
 
             min f'*x    subject to:  A*x  <= b
              x                       Aeq*x = beq
                                      lb <= x <= ub
                                      x(i) integer, where i is in the index
                                      vector intcon (integer constraints)
 
    X = INTLINPROG(f,intcon,A,b) solves the problem with integer variables
    in the intcon vector and linear inequality constraints A*x <= b. intcon
    is a vector of positive integers indicating components of the solution
    X that must be integers. For example, if you want to constrain X(2) and
    X(10) to be integers, set intcon to [2,10].
 
    X = INTLINPROG(f,intcon,A,b,Aeq,beq) solves the problem above while
    additionally satisfying the equality constraints Aeq*x = beq.
 
    X = INTLINPROG(f,intcon,A,b,Aeq,beq,LB,UB) defines a set of lower and
    upper bounds on the design variables, X, so that the solution is in the
    range LB <= X <= UB. Use empty matrices for LB and UB if no bounds
    exist. Set LB(i) = -Inf if X(i) is unbounded below; set UB(i) = Inf if
    X(i) is unbounded above.
 
    X = INTLINPROG(f,intcon,A,b,Aeq,beq,LB,UB,OPTIONS) minimizes with the
    default optimization parameters replaced by values in OPTIONS, an
    argument created with the OPTIMOPTIONS function. See OPTIMOPTIONS for
    details.
 
    X = INTLINPROG(PROBLEM) finds the minimum for PROBLEM. PROBLEM is a
    structure with the vector 'f' in PROBLEM.f, the integer constraints in
    PROBLEM.intcon, the linear inequality constraints in PROBLEM.Aineq and
    PROBLEM.bineq, the linear equality constraints in PROBLEM.Aeq and
    PROBLEM.beq, the lower bounds in PROBLEM.lb, the upper bounds in
    PROBLEM.ub, the options structure in PROBLEM.options, and solver name
    'intlinprog' in PROBLEM.solver.
 
    [X,FVAL] = INTLINPROG(f,intcon,A,b,...) returns the value of the
    objective function at X: FVAL = f'*X.
 
    [X,FVAL,EXITFLAG] = INTLINPROG(f,intcon,A,b,...) returns an EXITFLAG
    that describes the exit condition of INTLINPROG. Possible values of
    EXITFLAG and the corresponding exit conditions are
 
      2  Solver stopped prematurely. Integer feasible point found.
      1  Optimal solution found.
      0  Solver stopped prematurely. No integer feasible point found.
     -2  No feasible point found.
     -3  Root LP problem is unbounded.
 
    [X,FVAL,EXITFLAG,OUTPUT] = INTLINPROG(f,A,b,...) returns a structure
    OUTPUT containing information about the optimization process. OUTPUT
    includes the number of integer feasible points found and the final gap
    between internally calculated bounds on the solution. See the
    documentation for a complete description.
 
    See also LINPROG.

    Reference page in Help browser
       doc intlinprog

MILP Example 1


$$\begin{align} \min_{x} \quad & \begin{bmatrix} -3 & -1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \\ \text{subject to} \quad & \begin{bmatrix} 17 & 11 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \leq 83 \\ & \begin{bmatrix} 0 \\ 0 \end{bmatrix} \leq \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} \leq \begin{bmatrix} 3 \\ 7 \end{bmatrix} \\ & x_1, x_2 \in \mathbb{Z} \; \text{(integer)} \end{align} $$
In [5]:
f = [-3 -1]';
intcon = 1:2;
A = [17 11];
b = 83;
LB = [0 0]';
UB = [3 7]';

x = intlinprog(f,intcon,A,b,[],[],LB,UB)
Out[5]:
LP:                Optimal objective value is -11.909091.                                           

Cut Generation:    Applied 1 Gomory cut.                                                            
                   Lower bound is -11.000000.                                                       
                   Relative gap is 0.00%.                                                          


Optimal solution found.

Intlinprog stopped at the root node because the objective value is within a gap tolerance of the optimal value, options.TolGapAbs = 0
(the default value). The intcon variables are integer within tolerance, options.TolInteger = 1e-05 (the default value).


x =

    3.0000
    2.0000

MIP Example 2


$$\begin{align} \min_{x} \quad & \begin{bmatrix} -3 & -2 & -1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} \\ \text{subject to} \quad & \begin{bmatrix} 1 & 1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} \leq 7 \\ & \begin{bmatrix} 4 & 2 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = 12 \\ & 0 \leq \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} \\ & x_3 \in \{0,1\} \end{align} $$
In [6]:
f = [-3; -2; -1];
intcon = 3;
A = [1, 1, 1];
b = 7;
Aeq = [4,2,1];
beq = 12;
lb = zeros(3,1);
ub = [Inf;Inf;1];

x = intlinprog(f,intcon,A,b,Aeq,beq,lb,ub)
Out[6]:
LP:                Optimal objective value is -12.000000.                                           


Optimal solution found.

Intlinprog stopped at the root node because the objective value is within a gap tolerance of the optimal value, options.TolGapAbs = 0
(the default value). The intcon variables are integer within tolerance, options.TolInteger = 1e-05 (the default value).


x =

         0
    5.5000
    1.0000

3.1. Knapsack problems revisit

  • There is a container(knapsack) of capacity C = 20. Further more, there is a set 6 of objects. Each object has a weight and a value.
  • Determine the number of each item to include in a collection so that the total size is less than or equal to 20 and the total value is as large as possible.


items 1 2 3 4 5 6
weights 10 9 4 2 1 20
values 175 90 20 50 10 200


Question: Can we formulate this knapsack problem into a binary integer program?

$\Rightarrow$ decision variables $\quad x_{i} \in \{0,1\} \quad i = 1,\cdots,6$


$$\begin{align} \max_{x} \quad & 175x_1 + 90x_2 + 20x_3 + 50x_4 + 10x_5 + 200x_6 \\ \text{subject to} \quad & 10x_1 + 9x_2 + 4x_3 + 2x_4 + x_5 + 20x_6 \leq 20 \\ \end{align} $$


$\Rightarrow$ intlinprog in Matlab to solve

In [7]:
n = 6;  % # of items
weights = [10 9 4 2 1 20];
values = [175 90 20 50 10 200];
maxWeight = 20;

% in a form of MILP

f = -values;        % max
A = weights;
b = maxWeight;
intcon = 1:n;
lb = zeros(n,1);
ub = ones(n,1);

x = intlinprog(f,intcon,A,b,[],[],lb,ub)
Out[7]:
LP:                Optimal objective value is -305.000000.                                          

Cut Generation:    Applied 1 strong CG cut.                                                         
                   Lower bound is -275.000000.                                                      
                   Relative gap is 0.00%.                                                          


Optimal solution found.

Intlinprog stopped at the root node because the objective value is within a gap tolerance of the optimal value, options.TolGapAbs = 0
(the default value). The intcon variables are integer within tolerance, options.TolInteger = 1e-05 (the default value).


x =

    1.0000
    1.0000
         0
         0
    1.0000
         0

3.2. Factory and warehouse

  • A campany wants to $\begin{cases} \text{1. build new factory in Ulsan, Busan, or both} \\ \text{2. build new warehouse (only one has to be built)} \\ \text{3. warehouse must be built in a city of a new factory} \\ \end{cases} $
  • Available capital is 10. We want to maximize total profit.


Profit Cost to build
1 Build a factory in Ulsan 7 5
2 Build a factory in Busan 5 3
3 Build a warehouse in Ulsan 6 5
4 Build a warehouse in Busan 4 2


  • Form a binary integer programming to find the optimal locations of new factory and warehouse, and solve it using intlinprog in MATLAB.

$\quad\;$ (Hint: define binary decision variables, then use them to define objective function and constraints.)

In [8]:
f = -[7 5 6 4]';
intcon = [1 2 3 4];
A = [5 3 5 2;
    -1 -1 0 0;
    -1 0 1 0;
    0 -1 0 1];
b = [10;
    -1;
    0;
    0];

Aeq = [0 0 1 1];
beq = [1];

lb = [0 0 0 0]';
ub = [1 1 1 1]';
[x fval] = intlinprog(f,intcon,A,b,Aeq,beq,lb,ub)
Out[8]:
LP:                Optimal objective value is -16.000000.                                           


Optimal solution found.

Intlinprog stopped at the root node because the objective value is within a gap tolerance of the optimal value, options.TolGapAbs = 0
(the default value). The intcon variables are integer within tolerance, options.TolInteger = 1e-05 (the default value).


x =

    1.0000
    1.0000
         0
    1.0000


fval =

   -16

3.3. Shortest path

  • Binary integer programming can be applied to finding the shortest path in the graph.
  • Define the binary decision variables and form the binary integer programming. Then you can solve it using intlinprog command in MATLAB



In [9]:
f = [1 4 2 4 8 11 3]';
Aeq = [1 1 0 0 0 0 0
    1 0 -1 0 0 -1 0
    0 1 1 -1 -1 0 0
    0 0 0 1 0 0 -1
    0 0 0 0 1 1 1];
beq = [1 0 0 0 1]';

intcon = 1:7;
lb = zeros(7,1);
ub = ones(7,1);

x = intlinprog(f,intcon,[],[],Aeq,beq,lb,ub)
Out[9]:
LP:                Optimal objective value is 10.000000.                                            


Optimal solution found.

Intlinprog stopped at the root node because the objective value is within a gap tolerance of the optimal value, options.TolGapAbs = 0
(the default value). The intcon variables are integer within tolerance, options.TolInteger = 1e-05 (the default value).


x =

     1
     0
     1
     1
     0
     0
     1

3.4. Robot Path Design

  • A circular mobile robot is to be navigated. The robot should take a path from point A to point B, avoiding the polygonal obstacles (hatched regions).
  • Design a collision free-path the robot can take with the minimum length.