**Recurrent Neural Network
**

By Prof. Seungchul Lee

http://iai.postech.ac.kr/

Industrial AI Lab at POSTECH

http://iai.postech.ac.kr/

Industrial AI Lab at POSTECH

Table of Contents

- RNNs are a family of neural networks for processing sequential data

- Separate parameters for each value of the time index

- Cannot share statistical strength across different time indices

In [1]:

```
%%html
<center><iframe src="https://www.youtube.com/embed/oYglxfBtSQk?rel=0"
width="560" height="315" frameborder="0" allowfullscreen></iframe></center>
```

- Input at each time is a vector
- Each layer has many neurons
- Output layer too may have many neurons
- But will represent everything simple boxes
- Each box actually represents an entire layer with many units

- Something that happens today only affects the output of the system for $N$ days into the future
- $N$ is the width of the system

- Problem: Increasing the “history” makes the network more complex

- Required: Infinite response systems
- What happens today can continue to affect the output forever
- Possibly with weaker and weaker influence

- Recursive
- Output contains information about the entire past

- The state-space model

- This is a recurrent neural network
- State summarizes information about the entire past

- Single Hidden Layer RNN (Simplest State-Space Model)

- Multiple Recurrent Layer RNN