**Time-Series Analysis**

By Prof. Seungchul Lee

http://iai.postech.ac.kr/

Industrial AI Lab at POSTECH

http://iai.postech.ac.kr/

Industrial AI Lab at POSTECH

Table of Contents

Regression, Classification, Dimension Reduction,

Based on snapshot-type data

- Sequence matters

- What is a sequence?
- sentence
- medical signals
- speech waveform
- vibration measurement

- Sequence Modeling
- Most of the real-world data is time-series
- There are important bits to be considered
- Past events
- Relationship between events
- Causality
- Credit assignment

- Learning the structure and hierarchy

- Use the past and present observations to predict the future

We will focus on linear difference equations (LDE), a surprisingly rich topic both theoretically and practivally.

For example,

$$ y[0]=1,\quad y[1]=\frac{1}{2},\quad y[2]=\frac{1}{4},\quad \cdots $$or by closed-form expression,

$$y[n]=\left(\frac{1}{2}\right)^n,\quad n≥0 $$or with a difference equation and an initial condition,

$$y[n]=\frac{1}{2}y[n−1],\quad y[0]=1$$High order homogeneous LDE

$$y[n]=\alpha_1 y[n−1] + \alpha_2 y [n−2] + \cdots + \alpha_k y[n-k]$$A series is

*stationary*if there is no systematic change in mean and variance over time- Example: radio static

A series is

*non-stationary*if mean and variance change over time- Example: GDP, population, weather, etc.

Formally

- Augmented DickeyFuller test

Informally

- Auto-Correlation Function (ACF)
- Normal Quantile Plot (Q-Q plot)

Q-Q Plot

- Compare distribution of the residuals to normal
- Scatter plot of residual quantiles against normal
- Stationary data: quantiles match normal ($45^o$ line)
- Non-stationary data: quantiles do not match (points off $45^o$ line)

**Non-linear trends**

- For example, population may grow exponentially

**Seasonal trends**

Some series may exhibit seasonal trends

For example, weather pattern, employment, inflation, etc.

**Combining Linear, Quadratic, and Seasonal Trends**

- Some data may have a combintation of trends

One solution is to apply repeated differencing to the series

For example, first remove seasonal trend. Then remove linear trend

Inspect model fit by examining residuals Q-Q plot

- Anternatively, include both linear and cyclical trend terms into the model

- Most classifiers ignored the sequential aspects of data

- Consider a system which can occupy one of $N$ discrete states or categories $q_t \in \{S_1,S_2,\cdots,S_N\}$

- We are interested in stochastic systems, in which state evolution is random

- Any joint distribution can be factored into a series of conditional distributions

$$p(q_0,q_1,\cdots,q_T ) = p(q_0) \; p(q_1 \mid q_0) \; p( q_2 \mid q_1 q_0 ) \; p( q_3 \mid q_2 q_1 q_0 ) \cdots$$

$$p(q_0,q_1,\cdots,q_T ) = p(q_0) \; p(q_1 \mid q_0) \; p( q_2 \mid q_1 ) \; p( q_3 \mid q_2 ) \cdots$$

- (Assumption) for a Markov process, the next state depends only on the current state:

- More clearly

- Given current state, the past does not matter
- The state captures all relevant information from the history
- The state is a sufficient statistic of the future

For a Markov state 𝑠 and successor state 𝑠′, the state transition probability is defined by

State transition matrix $P$ defines transition probabilities from all states $s$ to all successor states $s'$.