Dynamic Programming


By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH

Source

  • By Prof. David J. Malan from Harvard University
  • By Prof. Erik Demaine from MIT online lecture

Table of Contents

1. Recursive Algorithm

  • one of the central ideas of computer science
  • depends on solutions to smaller instances of the same problem ( = subproblem)
  • function to call itself (it is impossible in the real world)
  • factorial example
    • $n! = n\cdot(n-1) \cdots 2\cdot 1$



In [1]:
%%html
<center><iframe width="560" height="315" src="https://www.youtube.com/embed/t4MSwiqfLaY" 
frameborder="0" allowfullscreen></iframe></center>

1.1. foo example

In [2]:
print ('{c}\n'.format(c="Hello class"))
Hello class

In [3]:
print('Hello class')
Hello class
In [4]:
def foo(str):
    print(str)
In [5]:
foo('Hello class')
Hello class
In [6]:
def foo_recursion(str):
    print(str)
    foo_recursion(str)
In [7]:
# Do not run. It falls into infinite loop.

# foo_recursion('Hello class')
In [8]:
# base case

def foo_recursion_correct(n):
    if n <= 0:
        return
    else:
        print('Hi \n')
        foo_recursion_correct(n-1)
In [9]:
foo_recursion_correct(4)
Hi 

Hi 

Hi 

Hi 

1.2. Factorial Example

  • factorial example
    • $n! = n\cdot(n-1) \cdots 2\cdot 1$
In [10]:
print(range(5))
range(0, 5)
In [11]:
n = 5

m = 1
for i in range(n):    
    m = m*(i+1)    

print(m)
120
In [12]:
def fac(n):
    if n == 1:
        return 1    
    else:
        return n*fac(n-1)
In [13]:
# recursive
fac(5)
Out[13]:
120

2. Dynamic Programming

  • Dynamic Programming: general, powerful algorithm design technique
  • Fibonacci numbers:
$$ \begin{align*} F_1 &= F_2 = 1 \\ F_n &= F_{n-1} + F_{n-2} \end{align*} $$

2.1. Naive Recursive Algorithm

$$ \begin{align*} \text{fib}(n):&\\ &\text{if } n \leq 2:\; f = 1\\ &\text{else}:\; f = \text{fib}(n-1) + \text{fib}(n-2)\\ &\text{return } f \end{align*}$$

it works, Is it good?



2.2. Memorized DP Algorithm

$$ \begin{align*} \text{memo }= [\;]&\\ \text{fib}(n):&\\ & \text{if $n$ in memo : return memo}[n]\\ \\ & \text{if } n\leq 2 :\; f = 1\\ & \text{else}: \;f = \text{fib}(n-1) + \text{fib}(n-2)\\ \\ &\text{memo}[n] = f\\ &\text{return } f \end{align*}$$

Benefit?

  • fib$(n)$ only recurses the first time it's called



  • memorize (remember) & re-use solutions to subproblems that helps solve the problem.

$\quad \implies$ DP $\simeq$ recursion + memorization

In [14]:
%%html
<center><iframe width="560" height="315"src="https://www.youtube.com/embed/OQ5jsbhAv_M" 
frameborder="0" allowfullscreen></iframe></center>

2.3. Examples

2.3.1. Fibonacci

In [15]:
# naive Fibonacci

def fib(n):
    if n <= 2:
        return 1    
    else:
        return fib(n-1) + fib(n-2)
In [16]:
fib(10)
Out[16]:
55
In [17]:
# Memorized DP Fibonacci

def mfib(memo, n):
    if memo[n-1] != 0:
        return memo[n-1]        
    elif n <= 2:
        memo[n-1] = 1
        return 1    
    else:
        memo[n-1] = mfib(memo, n-1) + mfib(memo, n-2)
        return memo[n-1]
In [18]:
import numpy as np

n = 10

memo = np.zeros(n)
mfib(memo, n)
Out[18]:
55.0
In [19]:
memo
Out[19]:
array([ 1.,  1.,  2.,  3.,  5.,  8., 13., 21., 34., 55.])
In [20]:
%timeit fib(30)
185 ms ± 2.57 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
In [21]:
memo = np.zeros(30)
In [22]:
%timeit mfib(memo, 30)
407 ns ± 14.4 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

2.3.2. Climbing a Stair

You are climbing a stair case. Each time you can either make 1 step, 2 steps, or 3 steps. How many distinct ways can you climb if the stairs has $n = 30$ steps?

In [23]:
def stair(memo, n):
    if memo[n-1] != 0:
        m = memo[n-1]
    elif n == 1:        
        m = 1
        memo[n-1] = m
    elif n == 2:
        m = 2
        memo[n-1] = m
    elif n == 3:
        m = 4
        memo[n-1] = m
    else:
        m = stair(memo, n-1) + stair(memo, n-2) + stair(memo, n-3)
    
    memo[n-1] = m
    return m
In [24]:
global memo

def stair(n):
    if memo[n-1] != 0:
        m = memo[n-1]
    elif n == 1:        
        m = 1
        memo[n-1] = m
    elif n == 2:
        m = 2
        memo[n-1] = m
    elif n == 3:
        m = 4
        memo[n-1] = m
    else:
        m = stair(n-1) + stair(n-2) + stair(n-3)
    
    memo[n-1] = m
    return m
In [25]:
n = 10

global memo
memo = np.zeros(n)
stair(n)
print(memo)
[  1.   2.   4.   7.  13.  24.  44.  81. 149. 274.]

2.3.3. Knapsack Problem using Dynamic Programming (DP)

From MIT's Introduction to Computer Science and Programming

Knapsack problem

  • burglar (or thief) can carry at most 20 kg (i.e., maximum capacity = 20)

  • Quickly decide which item to carry

items 1 2 3 4 5 6
weight 10 9 4 2 1 20
value 175 90 20 50 10 200

Approach

  1. guess (or trial and error)

  2. Exhaustive search if possible

  3. "smarter way" $\implies$ Recursive or dynamic programming

$$ \text{key ideas} = \text{original problem} \rightarrow \begin{cases} \text{subproblem} \rightarrow & \begin{cases} \text{subproblem} \rightarrow \\ \text{subproblem} \rightarrow \end{cases}\\ \text{subproblem} \rightarrow & \begin{cases} \text{subproblem} \rightarrow\\ \text{subproblem} \rightarrow\end{cases}\end{cases} $$



Suppose we have the following function:

$\quad$[value, taken] = chooseBest(items(0:5), maxWeight)

1) item 1 is not taken

$\quad$[v_1, t_1] = chooseBest(items(1:5), maxWeight)

2) item 1 is taken

$\quad$ [v_2, t_2] = chooseBest(items(1:5), maxWeight - weights(1))

$\quad \qquad \, \, $ v_2 = values(1) + v_2

$\qquad\quad \, \,$ t_2 = [items(1), t_2]

In [26]:
global weights
global values

def chooseBest(items, maxWeight):
    if len(items) == 0 or maxWeight <= 0:
        value = 0
        taken = []
    else:
        first = items[0]
        rest = items[1:]
        w = weights[first]
        
        # do not take the first item
        v1, t1 = chooseBest(rest, maxWeight)
                
        # do take the first item
        v2, t2 = chooseBest(rest, maxWeight - w)
        v2 = values[first] + v2
        t2 = [first] + t2
        
        if v2 >= v1 and maxWeight - w >= 0:
            value = v2
            taken = t2
        else:
            value = v1
            taken = t1
            
    return value, taken    
In [27]:
items = range(6)
weights = [10, 9, 4, 2, 1, 20]
values = [175, 90, 20, 50, 10, 200]

maxWeight = 20

chooseBest(items, maxWeight)
Out[27]:
(275, [0, 1, 4])
In [28]:
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')