Anomaly Detection


By Prof. Seungchul Lee
http://iailab.kaist.ac.kr/
Industrial AI Lab at KAIST

Table of Contents


1. Anomaly

  • Anomalies and outliers are essentially the same thing
    • Objects that are different from most other objects
    • Something that deviates from what is standard, or expected (one classification)


Causes of Anomalies

  • Data from different class of object or underlying mechanism

    • disease vs. non-disease
    • fraud vs. not fraud
  • Data measurement and collection errors

  • Natural variation

    • tails on a Gaussian distribution

$$P_Y(Y=y) = \frac{1}{\sqrt{2\pi}} \exp \left(-\frac{1}{2} y^2 \right)$$


Anomaly Detection

  • Finding outliers


Applications of Anomaly Detection

  • Security & Surveillance


  • Biomedical Applications


  • Industrial Damage Detection


  • Machinery Defects Diagnostics
    • Diagnosis of machinery conditions
    • Early alarm of malfunctioning


Difficulties with Anomaly Detection

  • Scarcity of Anomalies

    • It is not easy to get anomaly data, because anomaly rarely happens
    • Overfitting issue occurs when there is only small number of data
  • Diverse Types of Anomalies

    • There are so many causes of anomalies
    • At the training stage of neural network, we cannot have all possible anomalies as input data