Linear Discriminant Analysis (LDA) is a dimensionality reduction technique commonly used for supervised classification problems. The goal of LDA is to project the dataset onto a lower-dimensional space while maximizing the class separability.
LDA is very similar to Principal Component Analysis (PCA).
LDA can be performed in 5 steps:
- Compute the mean vectors for the different classes from the dataset.
- Compute the scatter matrices (in-between-class and within-class scatter matrices).
- Compute the eigenvectors and corresponding eigenvalues for the scatter matrices.
- Sort the eigenvectors by decreasing eigenvalues and choose k eigenvectors with the largest eigenvalues.
- Use this eigenvector matrix to transform the samples onto the new subspace.
Linear Discriminant Analysis (LDA) is like PCA, but it focuses on maximizing the seperability among the known categories.
Assumptions:
LDA is parametric - assumes Normal Distribution of data.
LDA assumes that each input have same variance.
Why Use Linear Discriminant Analysis?
Dimensionality Reduction
Feature Extraction
Handling Multiclass Problems
Reducing Overfitting
Applications:
image recognition, text classification, bioinformatics, face recognition
Numerical Problem: Conversion of 2D to 1D
References:
LDA clearly Explained - StatQuest
LDA clearly Explained - StatQuest
