Convolutional Neural Networks (CNN)
By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH
Table of Contents
0. Video LecturesĀ¶
InĀ [3]:
%%html
<center><iframe src="https://www.youtube.com/embed/PRFb4L9u7zI?rel=0"
width="560" height="315" frameborder="0" allowfullscreen></iframe></center>
1. ConvolutionĀ¶
1.1. 1D ConvolutionĀ¶
1.2. Convolution on Image (= Convolution in 2D)Ā¶
Filter (or Kernel)
- Modify or enhance an image by filtering
- Filter images to emphasize certain features or remove other features
Filtering includes smoothing, sharpening and edge enhancement
Discrete convolution can be viewed as element-wise multiplication by a matrix
How to find the right Kernels
We learn many different kernels that make specific effect on images
Letās apply an opposite approach
We are not designing the kernel, but are learning the kernel from data
Can learn feature extractor from data using a deep learning framework
2. Convolutional Neural Networks (CNN)Ā¶
2.1. Motivation: Learning Visual FeaturesĀ¶
The bird occupies a local area and looks the same in different parts of an image. We should construct neural networks which exploit these properties.
ANN structure for object detecion in image
- does not seem the best
- did not make use of the fact that we are dealing with images
- Spatial organization of the input is destroyed by flattening
- Locality: objects tend to have a local spatial support
- fully and convolutionally connected layer $\rightarrow$ locally and convolutionally connected layer
- __Translation invariance__: object appearance is independent of location - Weight sharing: untis connected to different locations have the same weights - We are not designing the kernel, but are learning the kernel from data - _i.e._ We are learning visual feature extractor from data
2.2. Convolutional OperatorĀ¶
Convolution of CNN
- Local connectivity
- Weight sharing
Typically have sparse interactions
Convolutional Neural Networks
- Simply neural networks that use the convolution in place of general matrix multiplication in at least one of their layers
- Multiple channels
- Multiple kernels
2.3 Stride and PaddingĀ¶
- Strides: increment step size for the convolution operator
- Reduces the size of the output map
- No stride and no padding
- Stride example with kernel size 3Ć3 and a stride of 2
- Padding: artificially fill borders of image
- Useful to keep spatial dimension constant across filters
- Useful with strides and large receptive fields
- Usually fill with 0s
2.5. PoolingĀ¶
- Compute a maximum value in a sliding window (max pooling)
- Reduce spatial resolution for faster computation
- Achieve invariance to any permutation inside one of the cell
- Pooling size : $2\times2$ for example
2.6. CNN for ClassificationĀ¶
- CONV and POOL layers output high-level features of input
- Fully connected layer uses these features for classifying input image
- Express output as probability of image belonging to a particular class
3.1. TrainingĀ¶
InĀ [1]:
import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
InĀ [2]:
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
InĀ [3]:
mnist = tf.keras.datasets.mnist
(train_x, train_y), (test_x, test_y) = mnist.load_data()
train_x, test_x = train_x/255.0, test_x/255.0
train_x = train_x.reshape((train_x.shape[0], 28, 28, 1))
test_x = test_x.reshape((test_x.shape[0], 28, 28, 1))
InĀ [4]:
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(32,
(3,3),
activation = 'relu',
padding = 'SAME',
input_shape = (28, 28, 1)),
tf.keras.layers.MaxPool2D((2,2)),
tf.keras.layers.Conv2D(64,
(3,3),
activation = 'relu',
padding = 'SAME',
input_shape = (14, 14, 32)),
tf.keras.layers.MaxPool2D((2,2)),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation = 'relu'),
tf.keras.layers.Dense(10, activation = 'softmax')
])
InĀ [5]:
model.compile(optimizer = 'adam',
loss = 'sparse_categorical_crossentropy',
metrics = ['accuracy'])
InĀ [6]:
model.fit(train_x, train_y)
Out[6]:
3.2. Testing or EvaluatingĀ¶
InĀ [7]:
test_loss, test_acc = model.evaluate(test_x, test_y)
print('loss = {}, Accuracy = {} %'.format(round(test_loss,2), round(test_acc*100)))
InĀ [8]:
test_img = test_x[np.random.choice(test_x.shape[0], 1)]
predict = model.predict_on_batch(test_img)
mypred = np.argmax(predict, axis = 1)
plt.figure(figsize = (12,5))
plt.subplot(1,2,1)
plt.imshow(test_img.reshape(28, 28), 'gray')
plt.axis('off')
plt.subplot(1,2,2)
plt.stem(predict[0])
plt.show()
print('Prediction : {}'.format(mypred[0]))
4. Lab: CNN with Tensorflow (Steel Surface Defects)Ā¶
- NEU steel surface defects example
- To classify defects images into 6 classes
Download NEU steel surface defects images and labels
4.1. TrainingĀ¶
InĀ [9]:
train_x, train_y = np.load('./data_files/NEU_train_imgs.npy'), np.load('./data_files/NEU_train_labels.npy')
test_x, test_y = np.load('./data_files/NEU_test_imgs.npy'), np.load('./data_files/NEU_test_labels.npy')
train_x, test_x = train_x/255.0, test_x/255.0
train_x = train_x.reshape((train_x.shape[0], 200, 200, 1))
test_x = test_x.reshape((test_x.shape[0], 200, 200, 1))
InĀ [10]:
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(32,
(3,3),
activation = 'relu',
padding = 'SAME',
input_shape = (200, 200, 1)),
tf.keras.layers.MaxPool2D((2,2)),
tf.keras.layers.Conv2D(64,
(3,3),
activation = 'relu',
padding = 'SAME',
input_shape = (100, 100, 32)),
tf.keras.layers.MaxPool2D((2,2)),
tf.keras.layers.Conv2D(128,
(3,3),
activation = 'relu',
padding = 'SAME',
input_shape = (50, 50, 64)),
tf.keras.layers.MaxPool2D((2,2)),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation = 'relu'),
tf.keras.layers.Dense(6, activation = 'softmax')
])
InĀ [11]:
model.compile(optimizer = 'adam',
loss = 'sparse_categorical_crossentropy',
metrics = ['accuracy'])
InĀ [12]:
model.fit(train_x, train_y, epochs = 10)
Out[12]:
4.2. Testing or EvaluatingĀ¶
InĀ [13]:
test_loss, test_acc = model.evaluate(test_x, test_y)
print('loss = {}, Accuracy = {} %'.format(round(test_loss,2), round(test_acc*100)))
InĀ [14]:
name = ['scratches', 'rolled-in scale', 'pitted surface', 'patches', 'inclusion', 'crazing']
idx = np.random.choice(test_x.shape[0], 1)
test_img = test_x[idx]
GT = test_y[idx]
predict = model.predict_on_batch(test_img)
mypred = np.argmax(predict, axis = 1)
plt.figure(figsize = (12,5))
plt.subplot(1,2,1)
plt.imshow(test_img.reshape(200, 200), 'gray')
plt.axis('off')
plt.subplot(1,2,2)
plt.stem(predict[0])
plt.show()
print('Prediction : {}'.format(name[mypred[0]]))
print('Ground Truth : {}'.format(name[GT[0]]))
InĀ [15]:
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')