Modern CNNs


By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH

Table of Contents

1. ImageNet

  • Human performance = 5.1%




1.1. LeNet

  • CNN = Convolutional Neural Networks = ConvNet
  • LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition.
  • All are still the basic components of modern ConvNets!


1.2. AlexNet

  • Simplified version of Krizhevsky, Alex, Sutskever, and Hinton. "Imagenet classification with deep convolutional neural networks." NIPS 2012

  • LeNet-style backbone, plus:

    • ReLU [Nair & Hinton 2010]
      • RevoLUtion of deep learning
      • Accelerate training
    • Dropout [Hinton et al 2012]
      • In-network ensembling
      • Reduce overfitting
    • Data augmentation
      • Label-preserving transformation
      • Reduce overfitting