What is Denoising Autoencoder? A Denoising Autoencoder is a type of autoencoder designed to solve the problem of having more hidden nodes than input nodes. Here is a simple structure of a Denoising...
Sparse Autoencoder
What is Sparse Autoencoder? A Sparse Autoencoder is a type of autoencoder used to extract more features (i.e., to have more hidden nodes) than input nodes. It is also effective in preventing overfi...
Contrastive Divergence
What is Contrastive Divergence? Because RBM is a undirected neural network, contrastive divergence is devised to adjust weights. Here are the input nodes and hidden nodes for example: RBM ca...
Restricted Boltzmann Machine (RBM)
What is Restricted Boltzmann Machine (RBM) RBM is devised to overcome the limitations of computation. The reason for the increased amount of computation is that there are too many nodes to calculat...
Long Short-Term Memory Networks(LSTM)
What is LSTM? LSTM is devised to solve the vanishing gradient problem in RNN. Here is a basic structural image of LSTM: The meanings of the symbols are shown below: One thing to notice is ...
Full Connection
What is Full Connection? To get predicted values from flattened values, we must pass through ANNs as follows: Thus, the full process of CNN will be:
Flattening
What is flattening? Flattening is the process of converting the pooled feature map created in pooling into the input layer of an ANN.
Pooling
What is Pooling? Without pooling, machines cannot recognize objects that differ only in angle or saturation as the same object. To solve this problem, we add a pooling layer in a CNN. Assume that...
Convolution Operation
What is Convolution Operation? The formula of convolution is: $(f \times g)(t)\ =\ \int_{-\infty}^{\infty}f(\tau)g(t\ -\ \tau)d\tau$ Convolution is the simple application of a filter, called a f...
Back Propagation
What is Backpropagation? Update the weights according to how much they are responsible for the error like: The learning rate determines how much we update the weights.