Transfer Learning
By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH
Table of Contents
0. Video LecturesĀ¶
InĀ [1]:
%%html
<center><iframe src="https://www.youtube.com/embed/n296VACHPws?start=1242&rel=0"
width="560" height="315" frameborder="0" allowfullscreen></iframe></center>
1. Pre-trained Model (VGG16)Ā¶
- Training a model on ImageNet from scratch takes days or weeks.
- Many models trained on ImageNet and their weights are publicly available!
- Transfer learning
- Use pre-trained weights, remove last layers to compute representations of images
- The network is used as a generic feature extractor
- Train a classification model from these features on a new classification task
- Pre- trained models can extract more general image features that can help identify edges, textures, shapes, and object composition
- Better than handcrafted feature extraction on natural images
1.1. Import LibraryĀ¶
InĀ [1]:
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
1.2. Load DataĀ¶
Download data files from here
InĀ [2]:
train_imgs = np.load('./data_files/target_images.npy')
train_labels = np.load('./data_files/target_labels.npy')
test_imgs = np.load('./data_files/test_images.npy')
test_labels = np.load('./data_files/test_labels.npy')
print(train_imgs.shape)
print(train_labels[0]) # one-hot-encoded 5 classes
# remove one-hot-encoding
train_labels = np.argmax(train_labels, axis = 1)
test_labels = np.argmax(test_labels, axis = 1)
InĀ [3]:
n_train = train_imgs.shape[0]
n_test = test_imgs.shape[0]
# very small dataset
print(n_train)
print(n_test)
InĀ [4]:
Dict = ['Hat','Cube','Card','Torch','Screw']
plt.figure(figsize = (15,10))
plt.subplot(2,3,1)
plt.imshow(train_imgs[1])
plt.title("Label : {}".format(Dict[train_labels[1]]))
plt.axis('off')
plt.subplot(2,3,2)
plt.imshow(train_imgs[2])
plt.title("Label : {}".format(Dict[train_labels[2]]))
plt.axis('off')
plt.subplot(2,3,3)
plt.imshow(train_imgs[3])
plt.title("Label : {}".format(Dict[train_labels[3]]))
plt.axis('off')
plt.subplot(2,3,4)
plt.imshow(train_imgs[18])
plt.title("Label : {}".format(Dict[train_labels[18]]))
plt.axis('off')
plt.subplot(2,3,5)
plt.imshow(train_imgs[25])
plt.title("Label : {}".format(Dict[train_labels[25]]))
plt.axis('off')
plt.show()
InĀ [5]:
model_type = tf.keras.applications.vgg16
base_model = model_type.VGG16()
base_model.trainable = False
base_model.summary()
1.4. Testing for Target DataĀ¶
InĀ [6]:
idx = np.random.randint(n_test)
pred = base_model.predict(test_imgs[idx].reshape(-1, 224, 224, 3))
label = model_type.decode_predictions(pred)[0]
print('%s (%.2f%%)' % (label[0][1], label[0][2]*100))
print('%s (%.2f%%)' % (label[1][1], label[1][2]*100))
print('%s (%.2f%%)' % (label[2][1], label[2][2]*100))
print('%s (%.2f%%)' % (label[3][1], label[3][2]*100))
print('%s (%.2f%%)' % (label[4][1], label[4][2]*100))
plt.figure(figsize = (6,6))
plt.imshow(test_imgs[idx])
plt.title("Label : {}".format(Dict[test_labels[idx]]))
plt.axis('off')
plt.show()
2. Learn From ScratchĀ¶
InĀ [7]:
model = tf.keras.models.Sequential([
tf.keras.layers.Conv2D(filters = 32,
kernel_size = (3,3),
activation = 'relu',
padding = 'SAME',
input_shape = (224, 224, 3)),
tf.keras.layers.MaxPool2D((2,2)),
tf.keras.layers.Conv2D(filters = 64,
kernel_size = (3,3),
activation = 'relu',
padding = 'SAME'),
tf.keras.layers.MaxPool2D((2,2)),
tf.keras.layers.Conv2D(filters = 64,
kernel_size = (3,3),
activation = 'relu',
padding = 'SAME'),
tf.keras.layers.MaxPool2D((2,2)),
tf.keras.layers.Conv2D(filters = 64,
kernel_size = (3,3),
activation = 'relu',
padding = 'SAME'),
tf.keras.layers.MaxPool2D((2,2)),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(units = 128, activation = 'relu'),
tf.keras.layers.Dense(units = 5, activation = 'softmax')
])
InĀ [8]:
model.compile(optimizer = 'adam',
loss = 'sparse_categorical_crossentropy',
metrics = 'accuracy')
InĀ [9]:
model.fit(train_imgs, train_labels, batch_size = 10, epochs = 10)
Out[9]:
InĀ [10]:
test_loss, test_acc = model.evaluate(test_imgs, test_labels)
3. Transfer LearningĀ¶
- We assume that these model parameters contain the knowledge learned from the source data set and that this knowledge will be equally applicable to the target data set.
- We will train the output layer from scratch, while the parameters of all remaining layers are fine tuned based on the parameters of the source model.
- Or initialize all weights from pre-trained model, then train them with target data
3.1. Pre-trained Weights, BiasesĀ¶
InĀ [11]:
vgg16_weights = base_model.get_weights()
3.2. Build a Transfer Learning ModelĀ¶
InĀ [12]:
# replace new and trainable classifier layer
fc2_layer = base_model.layers[-2].output
output = tf.keras.layers.Dense(units = 5, activation = 'softmax')(fc2_layer)
# define new model
model = tf.keras.Model(inputs = base_model.inputs, outputs = output)
InĀ [13]:
model.summary()
3.3. Define Loss and OptimizerĀ¶
InĀ [14]:
model.compile(optimizer = 'adam',
loss = 'sparse_categorical_crossentropy',
metrics = 'accuracy')
3.4. OptimizeĀ¶
InĀ [15]:
model.fit(train_imgs, train_labels, batch_size = 10, epochs = 10)
Out[15]:
3.5. Test and EvaluateĀ¶
InĀ [16]:
test_loss, test_acc = model.evaluate(test_imgs, test_labels)
InĀ [17]:
test_x = test_imgs[np.random.choice(n_test, 1)]
pred = np.argmax(model.predict(test_x))
plt.figure(figsize = (6,6))
plt.imshow(test_x.reshape(224, 224, 3))
plt.axis('off')
plt.show()
print('Prediction : {}'.format(Dict[pred]))
InĀ [18]:
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')