Linear Discriminant Analysis - LDA Explanation with Numerical Problem

 Linear Discriminant Analysis (LDA) is a dimensionality reduction technique commonly used for supervised classification problems. The goal of LDA is to project the dataset onto a lower-dimensional space while maximizing the class separability.

LDA is very similar to Principal Component Analysis (PCA).

LDA can be performed in 5 steps:

1. Compute the mean vectors for the different classes from the dataset.
2. Compute the scatter matrices (in-between-class and within-class scatter matrices).
3. Compute the eigenvectors and corresponding eigenvalues for the scatter matrices.
4. Sort the eigenvectors by decreasing eigenvalues and choose k eigenvectors with the largest eigenvalues.
5. Use this eigenvector matrix to transform the samples onto the new subspace.

 Linear Discriminant Analysis (LDA) is like PCA, but it focuses on maximizing the seperability among the known categories.

Assumptions:

LDA is parametric - assumes Normal Distribution of data.

LDA assumes that each input have same variance.

Why Use Linear Discriminant Analysis?

Dimensionality Reduction

Feature Extraction

Handling Multiclass Problems

Reducing Overfitting

Applications:
image recognition, text classification, bioinformatics, face recognition

Numerical Problem: Conversion of 2D to 1D

References:
LDA clearly Explained - StatQuest

LDA Code Implementation

LDA Paper