Transfer Learning
By Prof. Seungchul Lee
http://iailab.kaist.ac.kr/
Industrial AI Lab at KAIST
http://iailab.kaist.ac.kr/
Industrial AI Lab at KAIST
Table of Contents
1. Pre-trained Model (VGG16)¶
- Training a model on ImageNet from scratch takes days or weeks.
- Many models trained on ImageNet and their weights are publicly available!
- Transfer learning
- Use pre-trained weights, remove last layers to compute representations of images
- The network is used as a generic feature extractor
- Train a classification model from these features on a new classification task
- Pre- trained models can extract more general image features that can help identify edges, textures, shapes, and object composition
- Better than handcrafted feature extraction on natural images
1.1. Import Library¶
In [1]:
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
1.2. Load Data¶
In [2]:
from google.colab import drive
drive.mount('/content/drive')
In [3]:
# Change file paths if necessary
train_imgs = np.load('/content/drive/MyDrive/DL_Colab/DL_data/tranfer_learning_train_images.npy')
train_labels = np.load('/content/drive/MyDrive/DL_Colab/DL_data/tranfer_learning_train_labels.npy')
test_imgs = np.load('/content/drive/MyDrive/DL_Colab/DL_data/tranfer_learning_test_images.npy')
test_labels = np.load('/content/drive/MyDrive/DL_Colab/DL_data/tranfer_learning_test_labels.npy')
print(train_imgs.shape)
print(train_labels[0]) # one-hot-encoded 5 classes
# remove one-hot-encoding
train_labels = np.argmax(train_labels, axis = 1)
test_labels = np.argmax(test_labels, axis = 1)
In [4]:
n_train = train_imgs.shape[0]
n_test = test_imgs.shape[0]
# very small dataset
print(n_train)
print(n_test)
In [5]:
Dict = ['Hat','Cube','Card','Torch','Screw']
plt.figure(figsize = (8, 6))
plt.subplot(2,3,1)
plt.imshow(train_imgs[1])
plt.title("Label: {}".format(Dict[train_labels[1]]))
plt.axis('off')
plt.subplot(2,3,2)
plt.imshow(train_imgs[2])
plt.title("Label: {}".format(Dict[train_labels[2]]))
plt.axis('off')
plt.subplot(2,3,3)
plt.imshow(train_imgs[3])
plt.title("Label: {}".format(Dict[train_labels[3]]))
plt.axis('off')
plt.subplot(2,3,4)
plt.imshow(train_imgs[18])
plt.title("Label: {}".format(Dict[train_labels[18]]))
plt.axis('off')
plt.subplot(2,3,5)
plt.imshow(train_imgs[25])
plt.title("Label: {}".format(Dict[train_labels[25]]))
plt.axis('off')
plt.show()
1.3. Load VGG16 Model¶
In [6]:
model_type = tf.keras.applications.vgg16
base_model = model_type.VGG16()
base_model.trainable = False
base_model.summary()
1.4. Testing for Target Data¶
In [7]:
idx = np.random.randint(n_test)
pred = base_model.predict(test_imgs[idx].reshape(-1, 224, 224, 3), verbose = 0)
label = model_type.decode_predictions(pred)[0]
print('%s (%.2f%%)' % (label[0][1], label[0][2]*100))
print('%s (%.2f%%)' % (label[1][1], label[1][2]*100))
print('%s (%.2f%%)' % (label[2][1], label[2][2]*100))
print('%s (%.2f%%)' % (label[3][1], label[3][2]*100))
print('%s (%.2f%%)' % (label[4][1], label[4][2]*100))
plt.figure(figsize = (4, 4))
plt.imshow(test_imgs[idx])
plt.title("Label : {}".format(Dict[test_labels[idx]]))
plt.axis('off')
plt.show()
2. Transfer Learning¶
- We assume that these model parameters contain the knowledge learned from the source data set and that this knowledge will be equally applicable to the target data set.
- We will train the output layer from scratch, while the parameters of all remaining layers are fine tuned based on the parameters of the source model.
- Or initialize all weights from pre-trained model, then train them with target data
3.1. Pre-trained Weights, Biases¶
In [8]:
vgg16_weights = base_model.get_weights()
2.2. Build a Transfer Learning Model¶
In [9]:
# replace new and trainable classifier layer
fc2_layer = base_model.layers[-2].output
output = tf.keras.layers.Dense(units = 5, activation = 'softmax')(fc2_layer)
# define new model
TL_model = tf.keras.Model(inputs = base_model.inputs, outputs = output)
In [10]:
TL_model.summary()
2.3. Define Loss and Optimizer¶
In [11]:
TL_model.compile(optimizer = 'adam',
loss = 'sparse_categorical_crossentropy',
metrics = 'accuracy')
2.4. Optimize¶
In [12]:
TL_model.fit(train_imgs, train_labels, batch_size = 10, epochs = 10)
Out[12]:
2.5. Test and Evaluate¶
In [13]:
test_loss, test_acc = TL_model.evaluate(test_imgs, test_labels)
In [14]:
test_x = test_imgs[np.random.choice(n_test, 1)]
pred = np.argmax(TL_model.predict(test_x, verbose = 0))
plt.figure(figsize = (4, 4))
plt.imshow(test_x.reshape(224, 224, 3))
plt.axis('off')
plt.show()
print('Prediction : {}'.format(Dict[pred]))
In [15]:
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')