AI for Mechanical Engineering

Convolutional Neural Networks (CNN)

Problem 1: Sign Language Classification with CNN

As we have learned in the lectures, Convolutional Neural Networks (CNN) have the capability to classify images directly. This characteristic makes CNN highly versatile and applicable to a wide range of industries. In this particular task, we aim to employ CNN for the classification of sign language. This application will enable individuals to engage in direct communication with those who are hearing impaired, even if they are not familiar with sign language.

  1. Load and plot 6 random data points. You need to load a total of 4 files.
In [ ]:
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
In [ ]:
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
In [ ]:
# Change file paths if necessary

train_x = np.load('/content/drive/MyDrive/DL_Colab/DL_data/sign_language_train_img.npy')
train_y = np.load('/content/drive/MyDrive/DL_Colab/DL_data/sign_language_train_label.npy')

test_x = np.load('/content/drive/MyDrive/DL_Colab/DL_data/sign_language_test_img.npy')
test_y = np.load('/content/drive/MyDrive/DL_Colab/DL_data/sign_language_test_label.npy')
In [ ]:
# check the shape of data

print(train_x.shape)
print(train_y.shape)
print(test_x.shape)
print(test_y.shape)
(900, 100, 100, 1)
(900, 6)
(337, 100, 100, 1)
(337, 6)
In [ ]:
## your code here
#
No description has been provided for this image
  1. Design your CNN structure and train it with the training data.
  • input
  • filter size
  • pooling size
  • hidden layer
  • output
In [ ]:
## your code here
#
In [ ]:
model.compile(optimizer = 'adam',
              loss = 'categorical_crossentropy',
              metrics = ['accuracy'])
In [ ]:
model.fit(train_x, train_y, epochs = 20)
Epoch 1/20
29/29 [==============================] - 6s 157ms/step - loss: 32.3575 - accuracy: 0.2044
Epoch 2/20
29/29 [==============================] - 3s 110ms/step - loss: 4.9883 - accuracy: 0.4733
Epoch 3/20
29/29 [==============================] - 3s 95ms/step - loss: 2.0535 - accuracy: 0.6544
Epoch 4/20
29/29 [==============================] - 3s 94ms/step - loss: 1.2098 - accuracy: 0.7400
Epoch 5/20
29/29 [==============================] - 3s 101ms/step - loss: 0.8137 - accuracy: 0.8189
Epoch 6/20
29/29 [==============================] - 5s 170ms/step - loss: 0.8209 - accuracy: 0.8256
Epoch 7/20
29/29 [==============================] - 3s 94ms/step - loss: 0.5930 - accuracy: 0.8622
Epoch 8/20
29/29 [==============================] - 3s 94ms/step - loss: 0.3508 - accuracy: 0.9111
Epoch 9/20
29/29 [==============================] - 3s 95ms/step - loss: 0.3858 - accuracy: 0.8978
Epoch 10/20
29/29 [==============================] - 3s 120ms/step - loss: 0.3434 - accuracy: 0.9156
Epoch 11/20
29/29 [==============================] - 4s 143ms/step - loss: 0.2289 - accuracy: 0.9233
Epoch 12/20
29/29 [==============================] - 3s 95ms/step - loss: 0.2852 - accuracy: 0.9200
Epoch 13/20
29/29 [==============================] - 3s 94ms/step - loss: 0.2753 - accuracy: 0.9189
Epoch 14/20
29/29 [==============================] - 3s 95ms/step - loss: 0.2312 - accuracy: 0.9333
Epoch 15/20
29/29 [==============================] - 4s 147ms/step - loss: 0.1573 - accuracy: 0.9511
Epoch 16/20
29/29 [==============================] - 6s 203ms/step - loss: 0.1503 - accuracy: 0.9533
Epoch 17/20
29/29 [==============================] - 3s 98ms/step - loss: 0.2285 - accuracy: 0.9322
Epoch 18/20
29/29 [==============================] - 3s 94ms/step - loss: 0.1649 - accuracy: 0.9522
Epoch 19/20
29/29 [==============================] - 5s 160ms/step - loss: 0.0878 - accuracy: 0.9689
Epoch 20/20
29/29 [==============================] - 3s 104ms/step - loss: 0.0619 - accuracy: 0.9767
Out[ ]:
<keras.src.callbacks.History at 0x7caf404de950>
  1. Test your model. Calculate accuracy and plot a random image with its predicted and true label.
  • Note: test accuracy should be higher than 80%.
In [ ]:
## your code here
#
11/11 [==============================] - 1s 37ms/step - loss: 0.7458 - accuracy: 0.8783
loss = 0.7458071708679199, Accuracy = 87.83382773399353 %
In [ ]:
## your code here
#
1/1 [==============================] - 0s 105ms/step
True: 4
Predict: 4
No description has been provided for this image

Problem 2: Understanding the Feature Map for Each Layer

Let's build a CNN model that classifies steel surface defects.

  • In order for CNN to preceed with classification, training was carried out using a convolution filter.
  • Let's visualize the feature map for each convolutional layer to understand the CNN's decision boundary.
  • NEU steel surface defects example.
  • To classify defects images into 6 classes.


Download NEU steel surface defects images and labels

  1. Load and plot 3 random images for 6 classes.
In [ ]:
# Change file paths if necessary

train_x = np.load('/content/drive/MyDrive/DL_Colab/DL_data/NEU_train_imgs.npy')
train_y = np.load('/content/drive/MyDrive/DL_Colab/DL_data/NEU_train_labels.npy')

test_x = np.load('/content/drive/MyDrive/DL_Colab/DL_data/NEU_test_imgs.npy')
test_y = np.load('/content/drive/MyDrive/DL_Colab/DL_data/NEU_test_labels.npy')

print(train_x.shape)
print(train_y.shape)
print(test_x.shape)
print(test_y.shape)
(1500, 200, 200, 1)
(1500,)
(300, 200, 200, 1)
(300,)
In [ ]:
name = ['scratches', 'rolled-in scale', 'pitted surface', 'patches', 'inclusion', 'crazing']

plt.figure(figsize = (12,7))

## your code here
#