Regularization

What is Regularization?

Regularization penalizes the model to prevent it from overfitting. That is, the model increases its potential to generalize by giving up a perfect fit to the training data. If the weights have very large values, the function will look like this:

Types of Regularization

• L2 Regularization

• Data Augmentation

• Dropout

• DropBlock

• BatchNorm

Implementation

• Regularization in Keras - No Regularization
  

• Regularization in Keras - With Regularization
  

• Regularization in PyTorch - Part 1
  

• Regularization in PyTorch - Part 2