Feature Reduction via Generalized Uncorrelated Linear Discriminant Analysis

View/ Open
Date
2006Author
Ye, Jieping
Janardan, Ravi
Li, Qi
Park, Haesun
Metadata
Show full item recordAbstract
High-dimensional data appear in many applications of data mining, machine learning, and bioinformatics.
Feature reduction is commonly applied as a preprocessing step to overcome the curse
of dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for
feature reduction. The extracted features via ULDA were shown to be statistically uncorrelated,
which is desirable for many applications. In this paper, an algorithm called ULDA/QR is proposed to simplify the previous implementation
of ULDA. Then the ULDA/GSVD algorithm is proposed based on a novel optimization criterion, to
address the singularity problem which occurs in undersampled problems, where the data dimension
is larger than the data size. The criterion used is the regularized version of the one in ULDA/QR.
Surprisingly, our theoretical result shows that the solution to ULDA/GSVD is independent of
the value of the regularization parameter. Experimental results on various types of datasets are reported to show the effectiveness of the proposed algorithm and to compare it with other commonly
used feature reduction algorithms.