For your handwritten solution, scan or take a picture of them (you can write it in markdown if you want).
For your code, only .ipynb file will be graded.
Please compress all the files to make a single .zip file
Do not submit a printed version of your code. It will not be graded.
In this problem, we are going to compute the gradient using the chain rule and dynamic programming, and update the weights $\omega \rightarrow \omega^+$. After that, the weights are updated through 1 back-propagation and compared with the error before the update.
Neural Network Model
1/2 MSE
for calculation convenience.
For example, $E = \frac{1}{2}\sum(\text{target} - \text{output})^2$(1) [hand written] Write and calculate $z_1$, $z_2$, $h_1$, $h_2$, $z_3$, $z_4$, $\sigma_1$, $\sigma_2$, and $E_{\text{total}}$ of forward propagation.
(2) [hand written] update $\omega_5$, $\omega_6$, $\omega_7$, $\omega_8$ $\rightarrow$ $\omega_5^+$, $\omega_6^+$, $\omega_7^+$, $\omega_8^+$ of back-propagation.
(3) [hand written] update $\omega_1$, $\omega_2$, $\omega_3$, $\omega_4$ $\rightarrow$ $\omega_1^+$, $\omega_2^+$, $\omega_3^+$, $\omega_4^+$ of back-propagation.
(4) [hand written] Write and calculate $E_{\text{total}}$ with the updated weights, and compare it to the previous error.
You will do binary classification for nonlinearly seperable data using MLP. Plot the given data first.
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
%matplotlib inline
N = 200
M = 2*N
gamma = 0.01
G0 = np.random.multivariate_normal([0, 0], gamma*np.eye(2), N)
G1 = np.random.multivariate_normal([1, 1], gamma*np.eye(2), N)
G2 = np.random.multivariate_normal([0, 1], gamma*np.eye(2), N)
G3 = np.random.multivariate_normal([1, 0], gamma*np.eye(2), N)
train_X = np.vstack([G0, G1, G2, G3])
train_y = np.vstack([np.ones([M,1]), np.zeros([M,1])])
train_X = np.asmatrix(train_X)
train_y = np.asmatrix(train_y)
print(train_X.shape)
print(train_y.shape)
plt.figure(figsize = (8, 6))
plt.plot(train_X[:M,0], train_X[:M,1], 'b.', alpha = 0.4, label = 'A')
plt.plot(train_X[M:,0], train_X[M:,1], 'r.', alpha = 0.4, label = 'B')
plt.axis('equal')
plt.xlim([-1, 2]); plt.ylim([-1, 2]);
plt.grid(alpha = 0.15)
plt.legend(fontsize = 12)
plt.show()
(1) Design a perceptron model which has a single layer, and train it to show the accuracy.
model = tf.keras.models.Sequential([
## your code here
])
model.summary()
## Your code here
#
(2) Plot the classifier (decision boundary).
## Your code here
#
(3) What is the highest accuracy you can get? Discuss the result.
## write down your discussion here
#
#
(4) Modify a perceptron model which has 2 layers, and train it to show the accuracy.
## Your code here
model = tf.keras.models.Sequential([
## your code here
])
model.summary()
## Your code here
#
(5) Plot two linear classification boundaries in the input space.
## your code here
#
(6) Plot one linear classification boundary in z space (or values in the hidden layer).
## your code here
#
In this problem, you are asked to use TensorFlow to implement the linear regression algorithm. By doing this, we hope that you are getting familiar with the syntax of TensorFlow.
# Data Generation
m = 5000
data_x = np.linspace(-3, 3, m)
data_y = 0.8*data_x + 2 + np.random.randn(m)*0.3
plt.figure(figsize = (10,8))
plt.plot(data_x, data_y, '.', alpha = 0.4)
plt.axis('equal')
plt.show()
We will build the simplest ANN model as shown in the following figure in order to find the best line fit (i.e., linear regression) for the given training data set. Note that there is no hidden layer, and both the input and output layer have only one neuron.
(1) Define the AI model.
model = tf.keras.models.Sequential([
## your code here
])
model.summary()
## Your code here
#
(2) Find the estimated weight and bias from the trained model.
## Your code here
#
(3) Plot the linear regression.
## Your code here
#