Comparing PCA, PCR and PLS
When does supervised dimensionality reduction outperform variance-based representation?
Question
How do principal component analysis, principal component regression and partial least squares compare as empirical strategies for dimension reduction and regression?
Positioning
This is a laboratory experiment in multivariate statistics, not a major applied project.
Methods that look similar at the linear-algebra level can differ once they are used for prediction. The experiment isolates what each method does to the predictor space, and what that implies for regression, rather than presenting a single preferred pipeline.
Method
The comparison treats PCA, PCR and PLS as distinct responses to the same empirical question: how to reduce dimension when the goal is prediction, not only description.