The catalog description for Applied Multivariate Statistical Analysis (STAT 5730) is: "Covers the standard topics of multivariate analysis. It will cover the assumptions, limitations, and uses of basic techniques such as cluster analysis, principal components analysis, and factor analysis as well as how to implement these methods in R and SPSS. Instead of theoretical development, the focus will be on the intuitive understanding and applications of these methods to real data sets by the students." However, these notes will concentrate on a theoretical development and applications will be downplayed. These notes are meant for self-study and may be of interest to students taking Applied Multivariate Statistical Analysis, the prequisite for which is "By permission of instructor" (though I would suggest Mathematical Statistics 1 [STAT 4047/5047] and Theory of Matrices [MATH 5090] when approaching these notes).

Copies of the classnotes are on the internet in PDF format as given below. The notes and supplements may contain hyperlinks to posted webpages; the links appear in red fonts. The "Proofs of Theorems" files were prepared in Beamer. The "Printout of Proofs" are printable PDF files of the Beamer slides without the pauses. These notes have not been classroom tested and may have typographical errors.

1. Comparison of Batches.

• Chapter 1. Comparison of Batches. Chapter 1 notes
• 1.1. Boxplots. Section 1.1 notes
• 1.2. Histograms.
• 1.3. Kernel Densities.
• 1.4. Scatterplots.
• 1.5. Chernoff-Flury Faces.
• 1.6. Andrew's Curves.
• 1.7. Parallel Coordinates Plots.
• 1.8. Hexagon Plots.
• 1.9. Boston Housing.
• 1.10. Exercises.
• Study Guide 1.

2. A Short Excursion into Matrix Algebra.

• 2.1. Elementary Operations. Section 2.1 notes
• 2.2. Spectral Decompositions. Section 2.2 notes
• 2.3. Quadratic Forms. Section 2.3 notes
• Supplement. Proofs from Section 2.3. Beamer file of Section 2.3 proofs
• Supplement. Printouts of Proofs from Section 2.3. Print file of Section 2.3 proofs
• 2.4. Derivatives. Section 2.4 notes
• 2.5. Partitioned Matrices. Section 2.5 notes
• Supplement. Proofs from Section 2.5. Beamer file of Section 2.5 proofs
• Supplement. Printouts of Proofs from Section 2.5. Print file of Section 2.5 proofs
• 2.6. Geometrical Aspects. Section 2.6 notes
• Supplement. Proofs from Section 2.6. Beamer file of Section 2.6 proofs
• Supplement. Printouts of Proofs from Section 2.6. Print file of Section 2.6 proofs
• 2.7. Exercises.
• Study Guide 2

3. Moving to Higher Dimensions.

• 3.1. Covariance. Section 3.1 notes
• 3.2. Correlation. (partial) Section 3.2 notes
• 3.3. Summary Statistics.
• 3.4. Linear Model for Two Variables.
• 3.5. Simple Analysis of Variance.
• 3.6. Multiple Linear Model.
• 3.7. Boston Housing.
• 3.8. Exercises.
• Study Guide 3.

4. Multivariate Distributions.

• 4.1. Distributions and Density Function.
• 4.2. Moments and Characteristic Functions.
• 4.3. Transformations.
• 4.4. The Multinormal Distribution.
• 4.5. Sampling Distributions and Limit Theorems.
• 4.6. Heavy-Tailed Distributions.
• 4.7. Copulae.
• 4.8. Bootstrap.
• 4.9. Exercises.
• Study Guide 4.

5. Theory of the Multinomial.

• 5.1. Elementary Properties of the Multinormal.
• 5.2. The Wishart Distribution.
• 5.3. Hotelling's T²-Distribution.
• 5.4. Spherical and Elliptical Distributions.
• 5.5. Exercises.
• Study Guide 5.

6. Theory of Estimation.

• 6.1. The Likelihood Function.
• 6.2. The Cramer-Rao Lower Bound.
• 6.3. Exercises.
• Study Guide 6.

7. Hypothesis Testing.

• 7.1. Likelihood Ratio Test.
• 7.2. Linear Hypothesis.
• 7.3. Boston Housing.
• 7.4. Exercises.
• Study Guide 7.

8. Regression Models.

• 8.1. General ANOVA and ANCOVA Models.
• 8.2. Categorical Responses.
• 8.3. Exercises.
• Study Guide 8.

9. Variable Selection.

• 9.1. Lasso.
• 9.2. Elastic Net.
• 9.3. Group Lasso.
• 9.4. Exercises.
• Study Guide 9.

10. Decomposition of Data Matrices by Factors.

• 10.1. The Geometric Point of View.
• 10.2. Fitting the p-Dimensional Point Cloud.
• 10.3. Fitting the n-Dimensional Point Cloud.
• 10.4. Relations Between Subspaces.
• 10.5. Practical Computation.
• 10.6. Exercises.
• Study Guide 10.

11. Principal Components Analysis.

• 11.1. Standardized Linear Combination.
• 11.2. Principal Components in Practice.
• 11.3. Interpretation of the PCs.
• 11.4. Asymptotic Properties of the PCs.
• 11.5. Normalized Principal Components Analysis.
• 11.6. Principal Comonents as a Factorial Method.
• 11.7. Common Principal Components.
• 11.8. Boston Housing.
• 11.9. More Examples.
• 11.10. Exercises.
• Study Guide 11.

12. Factor Analysis.

• 12.1. The Orthogonal Factor Model.
• 12.2. Estimation of the Factor Model.
• 12.3. Factor Scores and Strategies.
• 12.4. Boston Housing.
• 12.5. Exercises.
• Study Guide 12.

13. Cluster Analysis.

• 13.1. The Problem.
• 13.2. The Proximity Between Objects.
• 13.3. Cluster Algorithms.
• 13.4. Boston Housing.
• 13.5. Exercises.
• Study Guide 13.

Additional Chapters:

• 14. Discriminant Analysis.
• 15. Correspondence Analysis.
• 16. Canonical Correlation Analysis.
• 17. Multidimensional Scaling.
• 18. Conjoint Measurement Analysis.
• 19. Application in Finance.
• 20. Computability Intensive Techniques.

• 21. Symbols and Notation.
• 22. Data.

Return to Bob Gardner's home page