Convolutional Neural Networks (CNN)
Problem 1: Sign Language Classification with CNN¶
As we have learned in the lectures, Convolutional Neural Networks (CNN) have the capability to classify images directly. This characteristic makes CNN highly versatile and applicable to a wide range of industries. In this particular task, we aim to employ CNN for the classification of sign language. This application will enable individuals to engage in direct communication with those who are hearing impaired, even if they are not familiar with sign language.
- Load and plot 6 random data points. You need to load a total of 4 files.
In [ ]:
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
In [ ]:
from google.colab import drive
drive.mount('/content/drive')
In [ ]:
# Change file paths if necessary
train_x = np.load('/content/drive/MyDrive/DL_Colab/DL_data/sign_language_train_img.npy')
train_y = np.load('/content/drive/MyDrive/DL_Colab/DL_data/sign_language_train_label.npy')
test_x = np.load('/content/drive/MyDrive/DL_Colab/DL_data/sign_language_test_img.npy')
test_y = np.load('/content/drive/MyDrive/DL_Colab/DL_data/sign_language_test_label.npy')
In [ ]:
# check the shape of data
print(train_x.shape)
print(train_y.shape)
print(test_x.shape)
print(test_y.shape)
In [ ]:
## your code here
#
- Design your CNN structure and train it with the training data.
- input
- filter size
- pooling size
- hidden layer
- output
In [ ]:
## your code here
#
In [ ]:
model.compile(optimizer = 'adam',
loss = 'categorical_crossentropy',
metrics = ['accuracy'])
In [ ]:
model.fit(train_x, train_y, epochs = 20)
Out[Â ]:
- Test your model. Calculate accuracy and plot a random image with its predicted and true label.
- Note: test accuracy should be higher than 80%.
In [ ]:
## your code here
#
In [ ]:
## your code here
#
Problem 2: Understanding the Feature Map for Each Layer¶
Let's build a CNN model that classifies steel surface defects.
- In order for CNN to preceed with classification, training was carried out using a convolution filter.
- Let's visualize the feature map for each convolutional layer to understand the CNN's decision boundary.
- NEU steel surface defects example.
- To classify defects images into 6 classes.
Download NEU steel surface defects images and labels
- Load and plot 3 random images for 6 classes.
In [ ]:
# Change file paths if necessary
train_x = np.load('/content/drive/MyDrive/DL_Colab/DL_data/NEU_train_imgs.npy')
train_y = np.load('/content/drive/MyDrive/DL_Colab/DL_data/NEU_train_labels.npy')
test_x = np.load('/content/drive/MyDrive/DL_Colab/DL_data/NEU_test_imgs.npy')
test_y = np.load('/content/drive/MyDrive/DL_Colab/DL_data/NEU_test_labels.npy')
print(train_x.shape)
print(train_y.shape)
print(test_x.shape)
print(test_y.shape)
In [ ]:
name = ['scratches', 'rolled-in scale', 'pitted surface', 'patches', 'inclusion', 'crazing']
plt.figure(figsize = (12,7))
## your code here
#
- Design your CNN structure and train it with the training data.
- input
- filter size
- pooling size
- hidden layer
- output
- Note:
- Check the data shape. (CNN input is image !)
- Check y-label shape. (one-hot encoding)
- Construct 5-convolution blocks (conv_layer and pool_layer)
In [ ]:
## your code here
#
In [ ]:
model.compile(optimizer = tf.keras.optimizers.Adam(0.001),
loss = 'sparse_categorical_crossentropy',
metrics = ['accuracy'])
In [ ]:
model.fit(train_x, train_y, epochs = 20)
Out[Â ]:
- Test your model. Compute accuracy and plot a random image with its predicted and true label.
- Note: test accuracy should be higher than 90%.
In [ ]:
## your code here
#
In [ ]:
## your code here
#
- Visualize the feature maps in convolutional layers 1 through 5.
- Use the first data from the test dataset.
- Visualize up to 6 channels per layer.
In [ ]:
outputs = [layer.output for layer in model.layers]
outputs
Out[Â ]:
In [ ]:
test_img = test_x[0]
plt.figure(figsize = (4,4))
plt.imshow(test_img.reshape(200, 200), 'gray')
plt.axis('off')
plt.show()
In [ ]:
## your code here
#
