Pre-trained CNNs
http://iailab.kaist.ac.kr/
Industrial AI Lab at KAIST
Table of Contents
1. ImageNet¶
- Human performance = 5.1%
1.1. LeNet¶
- CNN = Convolutional Neural Networks = ConvNet
- LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition.
- All are still the basic components of modern ConvNets!
1.2. AlexNet¶
Simplified version of Krizhevsky, Alex, Sutskever, and Hinton. "Imagenet classification with deep convolutional neural networks." NIPS 2012
LeNet-style backbone, plus:
- ReLU [Nair & Hinton 2010]
- RevoLUtion of deep learning
- Accelerate training
- Dropout [Hinton et al 2012]
- In-network ensembling
- Reduce overfitting
- Data augmentation
- Label-preserving transformation
- Reduce overfitting
- ReLU [Nair & Hinton 2010]
1.3. VGG-16/19¶
Simonyan, Karen, and Zisserman. "Very deep convolutional networks for large-scale image recognition." (2014)
Simply “Very Deep”!
- Modularized design
- 3x3 Conv as the module
- Stack the same module
- Same computation for each module
- Stage-wise training
- VGG-11 → VGG-13 → VGG-16
- We need a better initialization…
- Modularized design
1.4. GoogleNet/Inception¶
- Multiple branches
- e.g., 1x1, 3x3, 5x5, pool
- Shortcuts
- stand-alone 1x1, merged by concat.
- Bottleneck
- Reduce dim by 1x1 before expensive 3x3/5x5 conv
1.5. ResNet¶
- He, Kaiming, et al. "Deep residual learning for image recognition." CVPR. 2016.
Skip Connection and Residual Net
- A direct connection between 2 non-consecutive layers
No gradient vanishing
Parameters are optimized to learn a residual, that is the difference between the value before the block and the one needed after.
A skip connection is a connection that bypasses at least one layer.
- Here, it is often used to transfer local information by concatenating or summing feature maps from the downsampling path with feature maps from the upsampling path.
- Merging features from various resolution levels helps combining context information with spatial information.
def residual_net(x):
conv1 = tf.keras.layers.Conv2D(filters = 32,
kernel_size = (3, 3),
padding = "SAME",
activation = 'relu')(x)
conv2 = tf.keras.layers.Conv2D(filters = 32,
kernel_size = (3, 3),
padding = "SAME",
activation = 'relu')(conv1)
maxp2 = tf.keras.layers.MaxPool2D(pool_size = (2, 2),
strides = 2)(conv2 + x)
flat = tf.keras.layers.Flatten()(maxp2)
hidden = tf.keras.layers.Dense(units = n_hidden,
activation='relu')(flat)
output = tf.keras.layers.Dense(units = n_output)(hidden)
return output
1.6. DenseNets¶
1.7 U-Net¶
The U-Net owes its name to its symmetric shape
The U-Net architecture is built upon the Fully Convolutional Network and modified in a way that it yields better segmentation in medical imaging.
Compared to FCN-8, the two main differences are
- U-net is symmetric and
- the skip connections between the downsampling path and the upsampling path apply a concatenation operator instead of a sum.
These skip connections intend to provide local information to the global information while upsampling. Because of its symmetry, the network has a large number of feature maps in the upsampling path, which allows to transfer information.
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import cv2
%matplotlib inline
3. Model Selection¶
# model_type = tf.keras.applications.densenet
# model_type = tf.keras.applications.inception_resnet_v2
# model_type = tf.keras.applications.inception_v3
model_type = tf.keras.applications.mobilenet
# model_type = tf.keras.applications.mobilenet_v2
# model_type = tf.keras.applications.nasnet
# model_type = tf.keras.applications.resnet50
# model_type = tf.keras.applications.vgg16
# model_type = tf.keras.applications.vgg19
3.1. Model Summary¶
model = model_type.MobileNet() # Change Model (hint : use capital name)
model.summary()
from google.colab import drive
drive.mount('/content/drive')
# img = cv2.imread('/content/drive/MyDrive/DL_Colab/DL_data/ILSVRC2017_test_00000005.JPEG')
img = cv2.imread('/content/drive/MyDrive/DL_Colab/DL_data/ILSVRC2017_test_00005381.JPEG')
print(img.shape)
plt.figure(figsize = (6, 6))
plt.imshow(img)
plt.axis('off')
plt.show()
resized_img = cv2.resize(img, (224, 224)).reshape(1, 224, 224, 3)
plt.figure(figsize = (6, 6))
plt.imshow(resized_img[0])
plt.axis('off')
plt.show()
input_img = model_type.preprocess_input(resized_img)
pred = model.predict(input_img, verbose = 0)
label = model_type.decode_predictions(pred)[0]
print('%s (%.2f%%)\n' % (label[0][1], label[0][2]*100))
print('%s (%.2f%%)\n' % (label[1][1], label[1][2]*100))
print('%s (%.2f%%)\n' % (label[2][1], label[2][2]*100))
print('%s (%.2f%%)\n' % (label[3][1], label[3][2]*100))
print('%s (%.2f%%)\n' % (label[4][1], label[4][2]*100))
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')