Convolutional Neural Networks (CNN)


By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH

Table of Contents

0. Video LecturesĀ¶

InĀ [3]:
%%html 
<center><iframe src="https://www.youtube.com/embed/PRFb4L9u7zI?rel=0" 
width="560" height="315" frameborder="0" allowfullscreen></iframe></center>

1. ConvolutionĀ¶

1.1. 1D ConvolutionĀ¶


1.2. Convolution on Image (= Convolution in 2D)Ā¶

Filter (or Kernel)

  • Modify or enhance an image by filtering
  • Filter images to emphasize certain features or remove other features
  • Filtering includes smoothing, sharpening and edge enhancement

  • Discrete convolution can be viewed as element-wise multiplication by a matrix


How to find the right Kernels

  • We learn many different kernels that make specific effect on images

  • Letā€™s apply an opposite approach

  • We are not designing the kernel, but are learning the kernel from data

  • Can learn feature extractor from data using a deep learning framework

2. Convolutional Neural Networks (CNN)Ā¶

2.1. Motivation: Learning Visual FeaturesĀ¶


The bird occupies a local area and looks the same in different parts of an image. We should construct neural networks which exploit these properties.



  • ANN structure for object detecion in image

    • does not seem the best
    • did not make use of the fact that we are dealing with images
    • Spatial organization of the input is destroyed by flattening



  • Locality: objects tend to have a local spatial support
    • fully and convolutionally connected layer $\rightarrow$ locally and convolutionally connected layer



- __Translation invariance__: object appearance is independent of location - Weight sharing: untis connected to different locations have the same weights - We are not designing the kernel, but are learning the kernel from data - _i.e._ We are learning visual feature extractor from data

2.2. Convolutional OperatorĀ¶

Convolution of CNN

  • Local connectivity
  • Weight sharing
  • Typically have sparse interactions

  • Convolutional Neural Networks

    • Simply neural networks that use the convolution in place of general matrix multiplication in at least one of their layers
  • Multiple channels


  • Multiple kernels


2.3 Stride and PaddingĀ¶

  • Strides: increment step size for the convolution operator
    • Reduces the size of the output map
  • No stride and no padding


  • Stride example with kernel size 3Ɨ3 and a stride of 2


  • Padding: artificially fill borders of image
    • Useful to keep spatial dimension constant across filters
    • Useful with strides and large receptive fields
    • Usually fill with 0s


2.4. Nonlinear Activation FunctionĀ¶


2.5. PoolingĀ¶

  • Compute a maximum value in a sliding window (max pooling)
    • Reduce spatial resolution for faster computation
    • Achieve invariance to any permutation inside one of the cell


  • Pooling size : $2\times2$ for example

2.6. CNN for ClassificationĀ¶

  • CONV and POOL layers output high-level features of input
  • Fully connected layer uses these features for classifying input image
  • Express output as probability of image belonging to a particular class



3. Lab: CNN with TensorFlow (MNIST)Ā¶

  • MNIST example
  • To classify handwritten digits



3.1. TrainingĀ¶

InĀ [1]:
import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
InĀ [2]:
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
InĀ [3]:
mnist = tf.keras.datasets.mnist

(train_x, train_y), (test_x, test_y) = mnist.load_data()

train_x, test_x = train_x/255.0, test_x/255.0
train_x = train_x.reshape((train_x.shape[0], 28, 28, 1))
test_x = test_x.reshape((test_x.shape[0], 28, 28, 1))
InĀ [4]:
model = tf.keras.models.Sequential([
    tf.keras.layers.Conv2D(32, 
                           (3,3), 
                           activation = 'relu',
                           padding = 'SAME',
                           input_shape = (28, 28, 1)),
    
    tf.keras.layers.MaxPool2D((2,2)),
    
    tf.keras.layers.Conv2D(64, 
                           (3,3), 
                           activation = 'relu',
                           padding = 'SAME',
                           input_shape = (14, 14, 32)),
    
    tf.keras.layers.MaxPool2D((2,2)),
    
    tf.keras.layers.Flatten(),
    
    tf.keras.layers.Dense(128, activation = 'relu'),
    
    tf.keras.layers.Dense(10, activation = 'softmax')
])
InĀ [5]:
model.compile(optimizer = 'adam',
              loss = 'sparse_categorical_crossentropy',
              metrics = ['accuracy'])
InĀ [6]:
model.fit(train_x, train_y)
1875/1875 [==============================] - 17s 9ms/step - accuracy: 0.9622 - loss: 0.1252
Out[6]:
<tensorflow.python.keras.callbacks.History at 0x7fdf3c1e3d30>

3.2. Testing or EvaluatingĀ¶

InĀ [7]:
test_loss, test_acc = model.evaluate(test_x, test_y)

print('loss = {}, Accuracy = {} %'.format(round(test_loss,2), round(test_acc*100)))
313/313 [==============================] - 1s 4ms/step - accuracy: 0.9847 - loss: 0.0432
loss = 0.04, Accuracy = 98 %
InĀ [8]:
test_img = test_x[np.random.choice(test_x.shape[0], 1)]

predict = model.predict_on_batch(test_img)
mypred = np.argmax(predict, axis = 1)

plt.figure(figsize = (12,5))

plt.subplot(1,2,1)
plt.imshow(test_img.reshape(28, 28), 'gray')
plt.axis('off')
plt.subplot(1,2,2)
plt.stem(predict[0])
plt.show()

print('Prediction : {}'.format(mypred[0]))
Prediction : 3

4. Lab: CNN with Tensorflow (Steel Surface Defects)Ā¶

  • NEU steel surface defects example
  • To classify defects images into 6 classes



Download NEU steel surface defects images and labels

4.1. TrainingĀ¶

InĀ [9]:
train_x, train_y = np.load('./data_files/NEU_train_imgs.npy'), np.load('./data_files/NEU_train_labels.npy')
test_x, test_y = np.load('./data_files/NEU_test_imgs.npy'), np.load('./data_files/NEU_test_labels.npy')

train_x, test_x = train_x/255.0, test_x/255.0
train_x = train_x.reshape((train_x.shape[0], 200, 200, 1))
test_x = test_x.reshape((test_x.shape[0], 200, 200, 1))
InĀ [10]:
model = tf.keras.models.Sequential([
    tf.keras.layers.Conv2D(32, 
                           (3,3), 
                           activation = 'relu',
                           padding = 'SAME',
                           input_shape = (200, 200, 1)),
    
    tf.keras.layers.MaxPool2D((2,2)),
    
    tf.keras.layers.Conv2D(64, 
                           (3,3), 
                           activation = 'relu',
                           padding = 'SAME',
                           input_shape = (100, 100, 32)),
    
    tf.keras.layers.MaxPool2D((2,2)),
    
    tf.keras.layers.Conv2D(128, 
                           (3,3), 
                           activation = 'relu',
                           padding = 'SAME',
                           input_shape = (50, 50, 64)),
    
    tf.keras.layers.MaxPool2D((2,2)),
    
    tf.keras.layers.Flatten(),    
    tf.keras.layers.Dense(128, activation = 'relu'),    
    tf.keras.layers.Dense(6, activation = 'softmax')
])
InĀ [11]:
model.compile(optimizer = 'adam',
              loss = 'sparse_categorical_crossentropy',
              metrics = ['accuracy'])
InĀ [12]:
model.fit(train_x, train_y, epochs = 10)
Epoch 1/10
47/47 [==============================] - 26s 549ms/step - accuracy: 0.2960 - loss: 1.6325
Epoch 2/10
47/47 [==============================] - 26s 558ms/step - accuracy: 0.6913 - loss: 0.8057
Epoch 3/10
47/47 [==============================] - 26s 555ms/step - accuracy: 0.8327 - loss: 0.4881
Epoch 4/10
47/47 [==============================] - 26s 554ms/step - accuracy: 0.8613 - loss: 0.3659
Epoch 5/10
47/47 [==============================] - 26s 556ms/step - accuracy: 0.8680 - loss: 0.3623
Epoch 6/10
47/47 [==============================] - 26s 550ms/step - accuracy: 0.9187 - loss: 0.2161
Epoch 7/10
47/47 [==============================] - 26s 553ms/step - accuracy: 0.9207 - loss: 0.2127
Epoch 8/10
47/47 [==============================] - 26s 552ms/step - accuracy: 0.9420 - loss: 0.1791
Epoch 9/10
47/47 [==============================] - 26s 550ms/step - accuracy: 0.9367 - loss: 0.2014
Epoch 10/10
47/47 [==============================] - 26s 549ms/step - accuracy: 0.9493 - loss: 0.1468
Out[12]:
<tensorflow.python.keras.callbacks.History at 0x7fdf50973ac8>

4.2. Testing or EvaluatingĀ¶

InĀ [13]:
test_loss, test_acc = model.evaluate(test_x, test_y)

print('loss = {}, Accuracy = {} %'.format(round(test_loss,2), round(test_acc*100)))
10/10 [==============================] - 1s 71ms/step - accuracy: 0.9400 - loss: 0.1650
loss = 0.17, Accuracy = 94 %
InĀ [14]:
name = ['scratches', 'rolled-in scale', 'pitted surface', 'patches', 'inclusion', 'crazing']

idx = np.random.choice(test_x.shape[0], 1)
test_img = test_x[idx]
GT = test_y[idx]

predict = model.predict_on_batch(test_img)
mypred = np.argmax(predict, axis = 1)

plt.figure(figsize = (12,5))

plt.subplot(1,2,1)
plt.imshow(test_img.reshape(200, 200), 'gray')
plt.axis('off')
plt.subplot(1,2,2)
plt.stem(predict[0])
plt.show()

print('Prediction : {}'.format(name[mypred[0]]))
print('Ground Truth : {}'.format(name[GT[0]]))
Prediction : rolled-in scale
Ground Truth : rolled-in scale
InĀ [15]:
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')