Dimensionality Reduction

What is Dimensionality Reduction?

In classification, we worked with datasets composed of only two independent variables. We did so for two reasons:

• Because we needed two dimensions to better visualize how Machine Learning models worked (by plotting the prediction regions and the prediction boundary for each model).

• Because regardless of the original number of our independent variables, we can often end up with two independent variables by applying an appropriate Dimensionality Reduction technique.

There are two types of Dimensionality Reduction techniques:

• Feature Selection
  • Backward Elimination
    
  • Forward Selection
    
  • Bidirectional Elimination
    
  • Score Comparison
• Feature Extraction
  • Principal Component Analysis (PCA)
    
  • Linear Discriminant Analysis (LDA)
    
  • Kernel PCA
    

You can see my implementation on my Github