Machine Learning


By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH

Table of Contents

1. Introduction

1.1. Course Info

  • Machine learning
    • Linear algebra
    • Optimization
    • Statistical and probabilistic approaches
  • Python in class and assignments
    • use it a lot
    • Provide all necessary .py codes for a class

1.2. What is Machine Learning

  • Draw a meaningful conclusion, given a set of data (observation, measurement)

  • In 1959, Arthur Samuel defined machine learning as a "Field of study that gives computers the ability to learn without being explicitly programmed"

    • Often hand programming not possible
    • Solution? Get the computer to program itself, by showing it examples of the behavior we want! This is the learning approach of AI
    • Really, we write the structure of the program and the computer tunes many internal parameters
  • Many related terms:
    • Pattern recognition
    • Neural networks $\rightarrow$ Deep learning
    • Data mining
    • Adaptive control
    • Statistical modeling
    • Data analytics / data science
    • Artificial intelligence
    • Machine learning

(source: lecture video from The Machine Learning Summer School by Zoubin Ghahramani, Univ. of Cambridge)

1.3. Learning: the View from Different Fields

  • Engineering
    • Signal processing, system identification, adaptive and optimal control, information theory, robotics, …
  • Computer science
    • Artificial intelligence, computer vision, …
  • Statistics
    • Learning theory, data mining, learning and inference from data, …
  • Cognitive science and psychology
    • Perception, movement control, reinforcement learning, mathematical psychology, …
  • Economics
    • Decision theory, game theory, operational research, …

(source: lecture video from The Machine Learning Summer School by Zoubin Ghahramani, Univ. of Cambridge)

1.4. Course Roadmap

  • Supervised Learning
    • Regression
      • linear, nonlinear (kernel), ridge ($L_1$ norm regularization), lasso ($L_2$ norm regularization)
    • Classification
      • perceptron, logistic regression, SVM, Beysian classifier
  • Unsupervised Learning
    • Clustering
      • k-means, Gaussian Mixture Model
      • graph partitioning (spectral clustering)
  • Required tools
    • linear algebra
      • matrix
      • $Ax = b$
      • projection
      • eigen analysis
      • SVD
    • optimization
      • least squares
      • cvx, linprog, intlinprog
    • statistics
      • Law of large numbers, central limit theorem
      • correlation
    • probability
      • Random variable, Gaussian density distribution, conditional probability,
      • maximum likelihood, maximum a posterior (MAP), Beysian thinking



2. What Will We Cover?

  • I will show you some examples

2.1. Data Fitting or Approximation (Regression)

  • a statistical process for estimating the relationships among variables (source: wikipedia)





2.2. Classification

  • the problem of identifying to which of a set of categories (sub-populations) a new observation belongs, on the basis of a training set of data containing observations (or instances) whose category membership is known (source: wikipedia)



2.3. Gaussian Density Distribution for Probabilistic Approach

  • Conditional of a joint Gaussian is Gaussian



2.4. Machine Learning in Image Processing

  • Low rank approximation

  • Data compression



2.5. Handwritten Digit Recognition

  • famous classification problem



2.6. Face Recognition

  • famous classification problem



2.7. Dimension Reduction (Multiple Senses + Principal Components)

  • the process of reducing the number of random variables under consideration, and can be divided into feature selection and feature extraction.



Next Class

In [1]:
%%javascript
$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js')