Autoencoder

Table of Contents

1. Unsupervised Learning

Definition

• Unsupervised learning refers to most attempts to extract information from a distribution that do not require human labor to annotate example
• Main task is to find the 'best' representation of the data

Dimension Reduction

• Attempt to compress as much information as possible in a smaller representation
• Preserve as much information as possible while obeying some constraint aimed at keeping the representation simpler

2. Autoencoders

It is like 'deep learning version' of dimension reduction in unsupervised learning.

Definition

• An autoencoder is a neural network that is trained to attempt to copy its input to its output
• The network consists of two parts: an encoder and a decoder that produce a reconstruction

Encoder and Decoder

• Encoder function : $z = f(x)$
• Decoder function : $x = g(z)$
• We learn to set $g\left(f(x)\right) = x$