Machine Learning


By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH

Table of Contents

1. Introduction¶

1.1. Course Info¶

  • Machine learning
    • Linear algebra
    • Optimization
    • Statistical and probabilistic approaches
  • Python in class and assignments
    • use it a lot
    • Provide all necessary .py codes for a class

1.2. What is Machine Learning¶

  • Draw a meaningful conclusion, given a set of data (observation, measurement)

  • In 1959, Arthur Samuel defined machine learning as a "Field of study that gives computers the ability to learn without being explicitly programmed"

    • Often hand programming not possible
    • Solution? Get the computer to program itself, by showing it examples of the behavior we want! This is the learning approach of AI
    • Really, we write the structure of the program and the computer tunes many internal parameters
  • Many related terms:
    • Pattern recognition
    • Neural networks $\rightarrow$ Deep learning
    • Data mining
    • Adaptive control
    • Statistical modeling
    • Data analytics / data science
    • Artificial intelligence
    • Machine learning

(source: lecture video from The Machine Learning Summer School by Zoubin Ghahramani, Univ. of Cambridge)

1.3. Learning: the View from Different Fields¶

  • Engineering
    • Signal processing, system identification, adaptive and optimal control, information theory, robotics, …
  • Computer science
    • Artificial intelligence, computer vision, …
  • Statistics
    • Learning theory, data mining, learning and inference from data, …
  • Cognitive science and psychology
    • Perception, movement control, reinforcement learning, mathematical psychology, …
  • Economics
    • Decision theory, game theory, operational research, …

(source: lecture video from The Machine Learning Summer School by Zoubin Ghahramani, Univ. of Cambridge)

1.4. Course Roadmap¶

  • Supervised Learning
    • Regression
      • linear, nonlinear (kernel), ridge ($L_1$ norm regularization), lasso ($L_2$ norm regularization)
    • Classification
      • perceptron, logistic regression, SVM, Beysian classifier
  • Unsupervised Learning
    • Clustering
      • k-means, Gaussian Mixture Model
      • graph partitioning (spectral clustering)
  • Required tools
    • linear algebra
      • matrix
      • $Ax = b$
      • projection
      • eigen analysis
      • SVD
    • optimization
      • least squares
      • cvx, linprog, intlinprog
    • statistics
      • Law of large numbers, central limit theorem
      • correlation
    • probability
      • Random variable, Gaussian density distribution, conditional probability,
      • maximum likelihood, maximum a posterior (MAP), Beysian thinking